changelog bump

Signed-off-by: Brian S. Stephan <bss@incorporeal.org>

.reuse/dep5

@@ -0,0 +1,34 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+Upstream-Name: GP2040-CE Binary Tools
+Upstream-Contact: Brian S. Stephan <bss@incorporeal.org>
+Source: https://github.com/OpenStickCommunity/gp2040ce-binary-tools
+
+# Trivial files
+Files: .gitignore .gitattributes
+Copyright: © 2023 Brian S. Stephan <bss@incorporeal.org>
+License: GPL-3.0-or-later
+
+# Protobuf from GP2040-CE (MIT) and derived compilations
+Files: gp2040ce_bintools/proto_snapshot/*
+Copyright: (c) 2024 OpenStickCommunity (gp2040-ce.info)
+License: MIT
+
+# High level repo docs
+Files: *.md
+Copyright: © 2023 Brian S. Stephan <bss@incorporeal.org>
+License: GPL-3.0-or-later
+
+# Test data (GP2040-CE-derived)
+Files: tests/test-files/proto-files/*
+Copyright: (c) 2024 OpenStickCommunity (gp2040-ce.info)
+License: MIT
+
+# Test data
+Files: tests/test-files/*
+Copyright: © 2023 Brian S. Stephan <bss@incorporeal.org>
+License: GPL-3.0-or-later
+
+# Python packaging, scaffolding, and errata
+Files: pyproject.toml tox.ini requirements/*
+Copyright: © 2023 Brian S. Stephan <bss@incorporeal.org>
+License: GPL-3.0-or-later

CHANGELOG.md

@@ -0,0 +1,243 @@
+# CHANGELOG
+
+Included is a summary of changes to the project. For full details, especially on behind-the-scenes code changes and
+development tools, see the commit history.
+
+## v0.11.1
+
+### Miscellaneous
+
+* The storage size and subsequent tests have been updated to expect a 32 KB config section (rather than the previous 16
+  KB).
+
+## v0.11.0
+
+### Improvements
+
+* The usage of provided GP2040-CE config Protobuf files must be explicitly specified via the `-S` flag. This is in order
+  to not accidentally fall back to them and be confused by the results. The net effect is that in most situations, users
+  will have to provide either `-P PATH` or `-S`.
+* The testing of the above fallbacks covers the options far better than before now.
+
+### Miscellaneous
+
+* Version bumps, which brought about a couple updates to the tests.
+
+## v0.10.0
+
+### Features
+
+* The board dump can now be made in UF2 format.
+* Kind of a retrograde feature --- precompiled Python files are no longer shipped with the project, as that is not
+  recommended by upstream protobuf. The version bumps have potentially fixed the Windows issues that led to the
+  precompiled Python files (rather than just the protobuf snapshots) being shipped in the first place, so we'll stop
+  shipping them for now.
+
+### Improvements
+
+* The protobuf snapshot files have been updated for GP2040-CE v0.7.10.
+
+### Miscellaneous
+
+* The TUI "Save As" test was checking old vs. new outputs, which wouldn't make any sense once the latest output fills in
+  new defaults, etc. not present in the origin file, which is pretty normal, so the test was simplified.
+* twine was added in the dev dependencies, and version bumps done for both requirements files.
+
+## v0.9.0
+
+### Features
+
+* The TUI editor can read and save configs as JSON files, which should create another avenue for making configs
+  available and editable for humans.
+
+### Miscellaneous
+
+* The `--version` flag outputs a bit of information about the program and Python version.
+* The config protobuf snapshot has been updated for (what should be) GP2040-CE v0.7.9.
+* Requirements version bumps.
+* REUSE specification errata.
+
+## v0.8.3
+
+### Improvements
+
+* `summarize-gp2040ce` can now understand the segmented UF2 files written in v0.8.1.
+* `concatenate` has an added `--backup` flag, which will move an existing output file aside before writing the new
+  output.
+
+## v0.8.2
+
+### Bugfixes
+
+* UF2 files made of parts now have the proper block counts.
+
+## v0.8.1
+
+### Improvements
+
+* `concatenate` no longer writes a padded 2 MB binary (so 4 MB UF2), instead properly indexing the board/user configs
+  separately from the binary, leading to smaller UF2s.
+
+### Bugfixes
+
+* `summarize-gp2040ce` now properly reads UF2 files.
+
+## v0.8.0
+
+### Features
+
+* New command, `summarize-gp2040ce`, to get info about a board image or USB device. Details are limited at the moment,
+  but more will come as we need them.
+* `dump-config` can now dump the board config rather than only the user config.
+
+### Improvements
+
+* `visualize-config` can now read UF2 files in addition to raw binary files.
+    * `visualize-config` is the new name of what was formerly `visualize-storage`, to keep things consistent-ish.
+* `concatenate`'s output flag is now `--new-filename`.
+
+### Miscellaneous
+
+* Increased test coverage, especially in the TUI, to stay at 90% despite not being able to cover some USB stuff.
+* The repository is now compliant with the REUSE specification.
+    * This included moving the DCO and making the license a Markdown file, for cleanliness.
+* Some minor docs updates.
+* The SPDX descriptor is used in `pyproject.toml` as that displays better.
+
+## v0.7.0
+
+### Features
+
+* New configurations can be saved as .bin/.uf2 files via "Save As..." in the TUI editor. This allows for making files of
+  different configurations that can be applied on top of one another simply by dragging the tiny UF2 onto the device.
+  This is useful for backup purposes and might also be a handy way to apply different configurations in a networkless
+  environment.
+
+### Improvements
+
+* The GP2040-CE configuration structure has been updated to v0.7.8-RC2.
+* There's a small About screen now in the TUI, but I didn't get around to adding online help yet.
+* TUI tweaks, none of which are earthshattering.
+
+### Miscellaneous
+
+* The license has been changed to GPLv3 (or later).
+* Library/dependency version bumps, a couple times.
+
+## v0.6.0
+
+### Added
+
+* Preliminary support for reading and writing "board config"s in the GP2040-CE binary. This will replace the precompiled
+  configuration for a particular board by way of `BoardConfig.h` with a second protobuf section of the flash that is
+  read from when a user config is not corrupt. `dump-config` can read that section, and `concatenate` can now write it;
+  the latter especially is expected to eventually be used by the core project's build tools.
+* `concatenate` can write out to UF2 format, which is a more convenient file for end users, directly relevant to the
+  above goals.
+* Precompiled protobuf files are now included in the Python package, so users don't need a clone of the GP2040-CE
+  repository in order to use the tools. This also works around a bug with the protobuf tool on Windows, where it can't
+  currently dynamically compile .proto files.
+* Licensing/attribution errata has been added, and a DCO has been added to the repo.
+
+### Improved
+
+* `libusb` usage now ignores a `NotImplementedError` on Windows for something that doesn't make sense on that platform,
+  making operations over USB more likely to work properly on Windows.
+
+## v0.5.1
+
+### Added
+
+* A little description to the README of what gp2040ce-binary-tools version supports what GP2040-CE version, because...
+
+### Changed
+
+* The flash addresses/storage sizes account for the expanded size in GP2040-CE v0.7.5.
+* Renamed the "pico" module to "rp2040" for accuracy's sake.
+
+## v0.5.0
+
+### Added
+
+* New `dump-gp2040ce` tool to dump a whole GP2040-CE board, saving the need for picotool to do it.
+* Flag to `concatenate` to truncate/replace the firmwary binary's storage section with the specified config in the
+  output binary.
+* Flag to `concatenate` to write firmware + config over USB.
+* Ability for `edit-config` to start with an empty configuration, if invoked with a non-existent file or against a board
+  with config errors.
+
+### Fixes
+
+* Write corruption is seemingly addressed by erasing and writing at 4096 byte boundaries.
+* Missing children are now referred to properly in `edit-config`.
+* `dump-config` pretended the filename was optional; it isn't.
+
+## v0.4.0
+
+### Added
+
+* `edit-config` can now read and write a config directly over USB (BOOTSEL mode) rather than working on dumped files.
+* `visualize-storage` can also read the config directly from the USB device.
+* New `dump-config` tool to get the config from the USB device rather than relying on picotool.
+* A whole new module for interacting with the Pico over USB, accordingly.
+
+## v0.3.2
+
+### Added
+
+* pyproject.toml changes to support building a package and publishing it.
+* Accordingly, this is the first version published to pypi.org.
+
+## v0.3.1
+
+### Added
+
+* Support for adding repeated protobuf elements, such as the 1 to 3 additional profiles.
+* Support for "opening" an empty configuration file (as in, starting with a blank config).
+
+## v0.3.0
+
+### Added
+
+* New `edit-config` tool to view and edit a dump of a GP2040-CE's configuration section in a terminal UI and save it back
+  to its original file.
+* This comes with lots of supporting code for generating config footers and so on.
+* The config's CRC checksum is now checked as part of parsing.
+
+## v0.2.1
+
+### Fixed
+
+* `concatenate` assumed the storage file was padded to the full 8192 byte length, causing it to put serialized configs
+  in the wrong spot; it now supports either option.
+
+## v0.2.0
+
+### Added
+
+* New `concatenate` tool to combine a firmware file with a storage file.
+* `visualize-storage` option to output in JSON format.
+
+## v0.1.2
+
+### Added
+
+* `visualize-storage` option to find the config from a whole flash dump of the board, rather than just the config section.
+
+### Changed
+
+* The minimum Python version is 3.9 (rather than unspecified).
+
+## v0.1.1
+
+### Added
+
+* The config footer is used to find the protobuf config, rather than guessing at it.
+* Some debug logging.
+
+## v0.1.0
+
+### Added
+
+* New `visualize-storage` tool to view a file of a GP2040-CE board's protobuf configuration.
+* Documentation for the above, and where the storage lives on the board.

CONTRIBUTING.md

@@ -0,0 +1,67 @@
+# Contributing Guidelines
+
+gp2040ce-binary-tools is made available under the GPLv3 (or later). Contributions are welcome via pull requests.  This
+document outlines the process to get your contribution accepted.
+
+## Sign Offs/Custody of Contributions
+
+The [Developer Certificate of Origin (DCO)](https://developercertificate.org/), reproduced below, is a way for
+contributors to certify that they wrote or otherwise have the right to license their code contributions to the project.
+Contributors must sign-off that they adhere to these requirements by adding a `Signed-off-by` line to their commit
+message, and/or, for frequent contributors, by signing off on their entry in `MAINTAINERS.md`.
+
+```
+Developer Certificate of Origin
+Version 1.1
+
+Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+
+Developer's Certificate of Origin 1.1
+
+By making a contribution to this project, I certify that:
+
+(a) The contribution was created in whole or in part by me and I
+    have the right to submit it under the open source license
+    indicated in the file; or
+
+(b) The contribution is based upon previous work that, to the best
+    of my knowledge, is covered under an appropriate open source
+    license and I have the right under that license to submit that
+    work with modifications, whether created in whole or in part
+    by me, under the same open source license (unless I am
+    permitted to submit under a different license), as indicated
+    in the file; or
+
+(c) The contribution was provided directly to me by some other
+    person who certified (a), (b) or (c) and I have not modified
+    it.
+
+(d) I understand and agree that this project and the contribution
+    are public and that a record of the contribution (including all
+    personal information I submit with it, including my sign-off) is
+    maintained indefinitely and may be redistributed consistent with
+    this project or the open source license(s) involved.
+```
+
+This process is followed by a number of open source projects, most notably the Linux kernel. Here's the gist of it:
+
+```
+[Your normal Git commit message here.]
+
+Signed-off-by: Random J Developer <random@developer.example.org>
+```
+
+`git help commit` has more info on adding this:
+
+```
+-s, --signoff
+    Add Signed-off-by line by the committer at the end of the commit log
+    message. The meaning of a signoff depends on the project, but it typically
+    certifies that committer has the rights to submit this work under the same
+    license and agrees to a Developer Certificate of Origin (see
+    http://developercertificate.org/ for more information).
+```

LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2023 Open Stick Community
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.

LICENSE.md

@@ -0,0 +1,675 @@
+# GNU GENERAL PUBLIC LICENSE
+
+Version 3, 29 June 2007
+
+Copyright (C) 2007 Free Software Foundation, Inc.
+<https://fsf.org/>
+
+Everyone is permitted to copy and distribute verbatim copies of this
+license document, but changing it is not allowed.
+
+## Preamble
+
+The GNU General Public License is a free, copyleft license for
+software and other kinds of works.
+
+The licenses for most software and other practical works are designed
+to take away your freedom to share and change the works. By contrast,
+the GNU General Public License is intended to guarantee your freedom
+to share and change all versions of a program--to make sure it remains
+free software for all its users. We, the Free Software Foundation, use
+the GNU General Public License for most of our software; it applies
+also to any other work released this way by its authors. You can apply
+it to your programs, too.
+
+When we speak of free software, we are referring to freedom, not
+price. Our General Public Licenses are designed to make sure that you
+have the freedom to distribute copies of free software (and charge for
+them if you wish), that you receive source code or can get it if you
+want it, that you can change the software or use pieces of it in new
+free programs, and that you know you can do these things.
+
+To protect your rights, we need to prevent others from denying you
+these rights or asking you to surrender the rights. Therefore, you
+have certain responsibilities if you distribute copies of the
+software, or if you modify it: responsibilities to respect the freedom
+of others.
+
+For example, if you distribute copies of such a program, whether
+gratis or for a fee, you must pass on to the recipients the same
+freedoms that you received. You must make sure that they, too, receive
+or can get the source code. And you must show them these terms so they
+know their rights.
+
+Developers that use the GNU GPL protect your rights with two steps:
+(1) assert copyright on the software, and (2) offer you this License
+giving you legal permission to copy, distribute and/or modify it.
+
+For the developers' and authors' protection, the GPL clearly explains
+that there is no warranty for this free software. For both users' and
+authors' sake, the GPL requires that modified versions be marked as
+changed, so that their problems will not be attributed erroneously to
+authors of previous versions.
+
+Some devices are designed to deny users access to install or run
+modified versions of the software inside them, although the
+manufacturer can do so. This is fundamentally incompatible with the
+aim of protecting users' freedom to change the software. The
+systematic pattern of such abuse occurs in the area of products for
+individuals to use, which is precisely where it is most unacceptable.
+Therefore, we have designed this version of the GPL to prohibit the
+practice for those products. If such problems arise substantially in
+other domains, we stand ready to extend this provision to those
+domains in future versions of the GPL, as needed to protect the
+freedom of users.
+
+Finally, every program is threatened constantly by software patents.
+States should not allow patents to restrict development and use of
+software on general-purpose computers, but in those that do, we wish
+to avoid the special danger that patents applied to a free program
+could make it effectively proprietary. To prevent this, the GPL
+assures that patents cannot be used to render the program non-free.
+
+The precise terms and conditions for copying, distribution and
+modification follow.
+
+## TERMS AND CONDITIONS
+
+### 0. Definitions.
+
+"This License" refers to version 3 of the GNU General Public License.
+
+"Copyright" also means copyright-like laws that apply to other kinds
+of works, such as semiconductor masks.
+
+"The Program" refers to any copyrightable work licensed under this
+License. Each licensee is addressed as "you". "Licensees" and
+"recipients" may be individuals or organizations.
+
+To "modify" a work means to copy from or adapt all or part of the work
+in a fashion requiring copyright permission, other than the making of
+an exact copy. The resulting work is called a "modified version" of
+the earlier work or a work "based on" the earlier work.
+
+A "covered work" means either the unmodified Program or a work based
+on the Program.
+
+To "propagate" a work means to do anything with it that, without
+permission, would make you directly or secondarily liable for
+infringement under applicable copyright law, except executing it on a
+computer or modifying a private copy. Propagation includes copying,
+distribution (with or without modification), making available to the
+public, and in some countries other activities as well.
+
+To "convey" a work means any kind of propagation that enables other
+parties to make or receive copies. Mere interaction with a user
+through a computer network, with no transfer of a copy, is not
+conveying.
+
+An interactive user interface displays "Appropriate Legal Notices" to
+the extent that it includes a convenient and prominently visible
+feature that (1) displays an appropriate copyright notice, and (2)
+tells the user that there is no warranty for the work (except to the
+extent that warranties are provided), that licensees may convey the
+work under this License, and how to view a copy of this License. If
+the interface presents a list of user commands or options, such as a
+menu, a prominent item in the list meets this criterion.
+
+### 1. Source Code.
+
+The "source code" for a work means the preferred form of the work for
+making modifications to it. "Object code" means any non-source form of
+a work.
+
+A "Standard Interface" means an interface that either is an official
+standard defined by a recognized standards body, or, in the case of
+interfaces specified for a particular programming language, one that
+is widely used among developers working in that language.
+
+The "System Libraries" of an executable work include anything, other
+than the work as a whole, that (a) is included in the normal form of
+packaging a Major Component, but which is not part of that Major
+Component, and (b) serves only to enable use of the work with that
+Major Component, or to implement a Standard Interface for which an
+implementation is available to the public in source code form. A
+"Major Component", in this context, means a major essential component
+(kernel, window system, and so on) of the specific operating system
+(if any) on which the executable work runs, or a compiler used to
+produce the work, or an object code interpreter used to run it.
+
+The "Corresponding Source" for a work in object code form means all
+the source code needed to generate, install, and (for an executable
+work) run the object code and to modify the work, including scripts to
+control those activities. However, it does not include the work's
+System Libraries, or general-purpose tools or generally available free
+programs which are used unmodified in performing those activities but
+which are not part of the work. For example, Corresponding Source
+includes interface definition files associated with source files for
+the work, and the source code for shared libraries and dynamically
+linked subprograms that the work is specifically designed to require,
+such as by intimate data communication or control flow between those
+subprograms and other parts of the work.
+
+The Corresponding Source need not include anything that users can
+regenerate automatically from other parts of the Corresponding Source.
+
+The Corresponding Source for a work in source code form is that same
+work.
+
+### 2. Basic Permissions.
+
+All rights granted under this License are granted for the term of
+copyright on the Program, and are irrevocable provided the stated
+conditions are met. This License explicitly affirms your unlimited
+permission to run the unmodified Program. The output from running a
+covered work is covered by this License only if the output, given its
+content, constitutes a covered work. This License acknowledges your
+rights of fair use or other equivalent, as provided by copyright law.
+
+You may make, run and propagate covered works that you do not convey,
+without conditions so long as your license otherwise remains in force.
+You may convey covered works to others for the sole purpose of having
+them make modifications exclusively for you, or provide you with
+facilities for running those works, provided that you comply with the
+terms of this License in conveying all material for which you do not
+control copyright. Those thus making or running the covered works for
+you must do so exclusively on your behalf, under your direction and
+control, on terms that prohibit them from making any copies of your
+copyrighted material outside their relationship with you.
+
+Conveying under any other circumstances is permitted solely under the
+conditions stated below. Sublicensing is not allowed; section 10 makes
+it unnecessary.
+
+### 3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+
+No covered work shall be deemed part of an effective technological
+measure under any applicable law fulfilling obligations under article
+11 of the WIPO copyright treaty adopted on 20 December 1996, or
+similar laws prohibiting or restricting circumvention of such
+measures.
+
+When you convey a covered work, you waive any legal power to forbid
+circumvention of technological measures to the extent such
+circumvention is effected by exercising rights under this License with
+respect to the covered work, and you disclaim any intention to limit
+operation or modification of the work as a means of enforcing, against
+the work's users, your or third parties' legal rights to forbid
+circumvention of technological measures.
+
+### 4. Conveying Verbatim Copies.
+
+You may convey verbatim copies of the Program's source code as you
+receive it, in any medium, provided that you conspicuously and
+appropriately publish on each copy an appropriate copyright notice;
+keep intact all notices stating that this License and any
+non-permissive terms added in accord with section 7 apply to the code;
+keep intact all notices of the absence of any warranty; and give all
+recipients a copy of this License along with the Program.
+
+You may charge any price or no price for each copy that you convey,
+and you may offer support or warranty protection for a fee.
+
+### 5. Conveying Modified Source Versions.
+
+You may convey a work based on the Program, or the modifications to
+produce it from the Program, in the form of source code under the
+terms of section 4, provided that you also meet all of these
+conditions:
+
+-   a) The work must carry prominent notices stating that you modified
+    it, and giving a relevant date.
+-   b) The work must carry prominent notices stating that it is
+    released under this License and any conditions added under
+    section 7. This requirement modifies the requirement in section 4
+    to "keep intact all notices".
+-   c) You must license the entire work, as a whole, under this
+    License to anyone who comes into possession of a copy. This
+    License will therefore apply, along with any applicable section 7
+    additional terms, to the whole of the work, and all its parts,
+    regardless of how they are packaged. This License gives no
+    permission to license the work in any other way, but it does not
+    invalidate such permission if you have separately received it.
+-   d) If the work has interactive user interfaces, each must display
+    Appropriate Legal Notices; however, if the Program has interactive
+    interfaces that do not display Appropriate Legal Notices, your
+    work need not make them do so.
+
+A compilation of a covered work with other separate and independent
+works, which are not by their nature extensions of the covered work,
+and which are not combined with it such as to form a larger program,
+in or on a volume of a storage or distribution medium, is called an
+"aggregate" if the compilation and its resulting copyright are not
+used to limit the access or legal rights of the compilation's users
+beyond what the individual works permit. Inclusion of a covered work
+in an aggregate does not cause this License to apply to the other
+parts of the aggregate.
+
+### 6. Conveying Non-Source Forms.
+
+You may convey a covered work in object code form under the terms of
+sections 4 and 5, provided that you also convey the machine-readable
+Corresponding Source under the terms of this License, in one of these
+ways:
+
+-   a) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by the
+    Corresponding Source fixed on a durable physical medium
+    customarily used for software interchange.
+-   b) Convey the object code in, or embodied in, a physical product
+    (including a physical distribution medium), accompanied by a
+    written offer, valid for at least three years and valid for as
+    long as you offer spare parts or customer support for that product
+    model, to give anyone who possesses the object code either (1) a
+    copy of the Corresponding Source for all the software in the
+    product that is covered by this License, on a durable physical
+    medium customarily used for software interchange, for a price no
+    more than your reasonable cost of physically performing this
+    conveying of source, or (2) access to copy the Corresponding
+    Source from a network server at no charge.
+-   c) Convey individual copies of the object code with a copy of the
+    written offer to provide the Corresponding Source. This
+    alternative is allowed only occasionally and noncommercially, and
+    only if you received the object code with such an offer, in accord
+    with subsection 6b.
+-   d) Convey the object code by offering access from a designated
+    place (gratis or for a charge), and offer equivalent access to the
+    Corresponding Source in the same way through the same place at no
+    further charge. You need not require recipients to copy the
+    Corresponding Source along with the object code. If the place to
+    copy the object code is a network server, the Corresponding Source
+    may be on a different server (operated by you or a third party)
+    that supports equivalent copying facilities, provided you maintain
+    clear directions next to the object code saying where to find the
+    Corresponding Source. Regardless of what server hosts the
+    Corresponding Source, you remain obligated to ensure that it is
+    available for as long as needed to satisfy these requirements.
+-   e) Convey the object code using peer-to-peer transmission,
+    provided you inform other peers where the object code and
+    Corresponding Source of the work are being offered to the general
+    public at no charge under subsection 6d.
+
+A separable portion of the object code, whose source code is excluded
+from the Corresponding Source as a System Library, need not be
+included in conveying the object code work.
+
+A "User Product" is either (1) a "consumer product", which means any
+tangible personal property which is normally used for personal,
+family, or household purposes, or (2) anything designed or sold for
+incorporation into a dwelling. In determining whether a product is a
+consumer product, doubtful cases shall be resolved in favor of
+coverage. For a particular product received by a particular user,
+"normally used" refers to a typical or common use of that class of
+product, regardless of the status of the particular user or of the way
+in which the particular user actually uses, or expects or is expected
+to use, the product. A product is a consumer product regardless of
+whether the product has substantial commercial, industrial or
+non-consumer uses, unless such uses represent the only significant
+mode of use of the product.
+
+"Installation Information" for a User Product means any methods,
+procedures, authorization keys, or other information required to
+install and execute modified versions of a covered work in that User
+Product from a modified version of its Corresponding Source. The
+information must suffice to ensure that the continued functioning of
+the modified object code is in no case prevented or interfered with
+solely because modification has been made.
+
+If you convey an object code work under this section in, or with, or
+specifically for use in, a User Product, and the conveying occurs as
+part of a transaction in which the right of possession and use of the
+User Product is transferred to the recipient in perpetuity or for a
+fixed term (regardless of how the transaction is characterized), the
+Corresponding Source conveyed under this section must be accompanied
+by the Installation Information. But this requirement does not apply
+if neither you nor any third party retains the ability to install
+modified object code on the User Product (for example, the work has
+been installed in ROM).
+
+The requirement to provide Installation Information does not include a
+requirement to continue to provide support service, warranty, or
+updates for a work that has been modified or installed by the
+recipient, or for the User Product in which it has been modified or
+installed. Access to a network may be denied when the modification
+itself materially and adversely affects the operation of the network
+or violates the rules and protocols for communication across the
+network.
+
+Corresponding Source conveyed, and Installation Information provided,
+in accord with this section must be in a format that is publicly
+documented (and with an implementation available to the public in
+source code form), and must require no special password or key for
+unpacking, reading or copying.
+
+### 7. Additional Terms.
+
+"Additional permissions" are terms that supplement the terms of this
+License by making exceptions from one or more of its conditions.
+Additional permissions that are applicable to the entire Program shall
+be treated as though they were included in this License, to the extent
+that they are valid under applicable law. If additional permissions
+apply only to part of the Program, that part may be used separately
+under those permissions, but the entire Program remains governed by
+this License without regard to the additional permissions.
+
+When you convey a copy of a covered work, you may at your option
+remove any additional permissions from that copy, or from any part of
+it. (Additional permissions may be written to require their own
+removal in certain cases when you modify the work.) You may place
+additional permissions on material, added by you to a covered work,
+for which you have or can give appropriate copyright permission.
+
+Notwithstanding any other provision of this License, for material you
+add to a covered work, you may (if authorized by the copyright holders
+of that material) supplement the terms of this License with terms:
+
+-   a) Disclaiming warranty or limiting liability differently from the
+    terms of sections 15 and 16 of this License; or
+-   b) Requiring preservation of specified reasonable legal notices or
+    author attributions in that material or in the Appropriate Legal
+    Notices displayed by works containing it; or
+-   c) Prohibiting misrepresentation of the origin of that material,
+    or requiring that modified versions of such material be marked in
+    reasonable ways as different from the original version; or
+-   d) Limiting the use for publicity purposes of names of licensors
+    or authors of the material; or
+-   e) Declining to grant rights under trademark law for use of some
+    trade names, trademarks, or service marks; or
+-   f) Requiring indemnification of licensors and authors of that
+    material by anyone who conveys the material (or modified versions
+    of it) with contractual assumptions of liability to the recipient,
+    for any liability that these contractual assumptions directly
+    impose on those licensors and authors.
+
+All other non-permissive additional terms are considered "further
+restrictions" within the meaning of section 10. If the Program as you
+received it, or any part of it, contains a notice stating that it is
+governed by this License along with a term that is a further
+restriction, you may remove that term. If a license document contains
+a further restriction but permits relicensing or conveying under this
+License, you may add to a covered work material governed by the terms
+of that license document, provided that the further restriction does
+not survive such relicensing or conveying.
+
+If you add terms to a covered work in accord with this section, you
+must place, in the relevant source files, a statement of the
+additional terms that apply to those files, or a notice indicating
+where to find the applicable terms.
+
+Additional terms, permissive or non-permissive, may be stated in the
+form of a separately written license, or stated as exceptions; the
+above requirements apply either way.
+
+### 8. Termination.
+
+You may not propagate or modify a covered work except as expressly
+provided under this License. Any attempt otherwise to propagate or
+modify it is void, and will automatically terminate your rights under
+this License (including any patent licenses granted under the third
+paragraph of section 11).
+
+However, if you cease all violation of this License, then your license
+from a particular copyright holder is reinstated (a) provisionally,
+unless and until the copyright holder explicitly and finally
+terminates your license, and (b) permanently, if the copyright holder
+fails to notify you of the violation by some reasonable means prior to
+60 days after the cessation.
+
+Moreover, your license from a particular copyright holder is
+reinstated permanently if the copyright holder notifies you of the
+violation by some reasonable means, this is the first time you have
+received notice of violation of this License (for any work) from that
+copyright holder, and you cure the violation prior to 30 days after
+your receipt of the notice.
+
+Termination of your rights under this section does not terminate the
+licenses of parties who have received copies or rights from you under
+this License. If your rights have been terminated and not permanently
+reinstated, you do not qualify to receive new licenses for the same
+material under section 10.
+
+### 9. Acceptance Not Required for Having Copies.
+
+You are not required to accept this License in order to receive or run
+a copy of the Program. Ancillary propagation of a covered work
+occurring solely as a consequence of using peer-to-peer transmission
+to receive a copy likewise does not require acceptance. However,
+nothing other than this License grants you permission to propagate or
+modify any covered work. These actions infringe copyright if you do
+not accept this License. Therefore, by modifying or propagating a
+covered work, you indicate your acceptance of this License to do so.
+
+### 10. Automatic Licensing of Downstream Recipients.
+
+Each time you convey a covered work, the recipient automatically
+receives a license from the original licensors, to run, modify and
+propagate that work, subject to this License. You are not responsible
+for enforcing compliance by third parties with this License.
+
+An "entity transaction" is a transaction transferring control of an
+organization, or substantially all assets of one, or subdividing an
+organization, or merging organizations. If propagation of a covered
+work results from an entity transaction, each party to that
+transaction who receives a copy of the work also receives whatever
+licenses to the work the party's predecessor in interest had or could
+give under the previous paragraph, plus a right to possession of the
+Corresponding Source of the work from the predecessor in interest, if
+the predecessor has it or can get it with reasonable efforts.
+
+You may not impose any further restrictions on the exercise of the
+rights granted or affirmed under this License. For example, you may
+not impose a license fee, royalty, or other charge for exercise of
+rights granted under this License, and you may not initiate litigation
+(including a cross-claim or counterclaim in a lawsuit) alleging that
+any patent claim is infringed by making, using, selling, offering for
+sale, or importing the Program or any portion of it.
+
+### 11. Patents.
+
+A "contributor" is a copyright holder who authorizes use under this
+License of the Program or a work on which the Program is based. The
+work thus licensed is called the contributor's "contributor version".
+
+A contributor's "essential patent claims" are all patent claims owned
+or controlled by the contributor, whether already acquired or
+hereafter acquired, that would be infringed by some manner, permitted
+by this License, of making, using, or selling its contributor version,
+but do not include claims that would be infringed only as a
+consequence of further modification of the contributor version. For
+purposes of this definition, "control" includes the right to grant
+patent sublicenses in a manner consistent with the requirements of
+this License.
+
+Each contributor grants you a non-exclusive, worldwide, royalty-free
+patent license under the contributor's essential patent claims, to
+make, use, sell, offer for sale, import and otherwise run, modify and
+propagate the contents of its contributor version.
+
+In the following three paragraphs, a "patent license" is any express
+agreement or commitment, however denominated, not to enforce a patent
+(such as an express permission to practice a patent or covenant not to
+sue for patent infringement). To "grant" such a patent license to a
+party means to make such an agreement or commitment not to enforce a
+patent against the party.
+
+If you convey a covered work, knowingly relying on a patent license,
+and the Corresponding Source of the work is not available for anyone
+to copy, free of charge and under the terms of this License, through a
+publicly available network server or other readily accessible means,
+then you must either (1) cause the Corresponding Source to be so
+available, or (2) arrange to deprive yourself of the benefit of the
+patent license for this particular work, or (3) arrange, in a manner
+consistent with the requirements of this License, to extend the patent
+license to downstream recipients. "Knowingly relying" means you have
+actual knowledge that, but for the patent license, your conveying the
+covered work in a country, or your recipient's use of the covered work
+in a country, would infringe one or more identifiable patents in that
+country that you have reason to believe are valid.
+
+If, pursuant to or in connection with a single transaction or
+arrangement, you convey, or propagate by procuring conveyance of, a
+covered work, and grant a patent license to some of the parties
+receiving the covered work authorizing them to use, propagate, modify
+or convey a specific copy of the covered work, then the patent license
+you grant is automatically extended to all recipients of the covered
+work and works based on it.
+
+A patent license is "discriminatory" if it does not include within the
+scope of its coverage, prohibits the exercise of, or is conditioned on
+the non-exercise of one or more of the rights that are specifically
+granted under this License. You may not convey a covered work if you
+are a party to an arrangement with a third party that is in the
+business of distributing software, under which you make payment to the
+third party based on the extent of your activity of conveying the
+work, and under which the third party grants, to any of the parties
+who would receive the covered work from you, a discriminatory patent
+license (a) in connection with copies of the covered work conveyed by
+you (or copies made from those copies), or (b) primarily for and in
+connection with specific products or compilations that contain the
+covered work, unless you entered into that arrangement, or that patent
+license was granted, prior to 28 March 2007.
+
+Nothing in this License shall be construed as excluding or limiting
+any implied license or other defenses to infringement that may
+otherwise be available to you under applicable patent law.
+
+### 12. No Surrender of Others' Freedom.
+
+If conditions are imposed on you (whether by court order, agreement or
+otherwise) that contradict the conditions of this License, they do not
+excuse you from the conditions of this License. If you cannot convey a
+covered work so as to satisfy simultaneously your obligations under
+this License and any other pertinent obligations, then as a
+consequence you may not convey it at all. For example, if you agree to
+terms that obligate you to collect a royalty for further conveying
+from those to whom you convey the Program, the only way you could
+satisfy both those terms and this License would be to refrain entirely
+from conveying the Program.
+
+### 13. Use with the GNU Affero General Public License.
+
+Notwithstanding any other provision of this License, you have
+permission to link or combine any covered work with a work licensed
+under version 3 of the GNU Affero General Public License into a single
+combined work, and to convey the resulting work. The terms of this
+License will continue to apply to the part which is the covered work,
+but the special requirements of the GNU Affero General Public License,
+section 13, concerning interaction through a network will apply to the
+combination as such.
+
+### 14. Revised Versions of this License.
+
+The Free Software Foundation may publish revised and/or new versions
+of the GNU General Public License from time to time. Such new versions
+will be similar in spirit to the present version, but may differ in
+detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program
+specifies that a certain numbered version of the GNU General Public
+License "or any later version" applies to it, you have the option of
+following the terms and conditions either of that numbered version or
+of any later version published by the Free Software Foundation. If the
+Program does not specify a version number of the GNU General Public
+License, you may choose any version ever published by the Free
+Software Foundation.
+
+If the Program specifies that a proxy can decide which future versions
+of the GNU General Public License can be used, that proxy's public
+statement of acceptance of a version permanently authorizes you to
+choose that version for the Program.
+
+Later license versions may give you additional or different
+permissions. However, no additional obligations are imposed on any
+author or copyright holder as a result of your choosing to follow a
+later version.
+
+### 15. Disclaimer of Warranty.
+
+THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
+APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
+HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT
+WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND
+PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE
+DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR
+CORRECTION.
+
+### 16. Limitation of Liability.
+
+IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
+WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR
+CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES,
+INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES
+ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT
+NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR
+LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM
+TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER
+PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+### 17. Interpretation of Sections 15 and 16.
+
+If the disclaimer of warranty and limitation of liability provided
+above cannot be given local legal effect according to their terms,
+reviewing courts shall apply local law that most closely approximates
+an absolute waiver of all civil liability in connection with the
+Program, unless a warranty or assumption of liability accompanies a
+copy of the Program in return for a fee.
+
+END OF TERMS AND CONDITIONS
+
+## How to Apply These Terms to Your New Programs
+
+If you develop a new program, and you want it to be of the greatest
+possible use to the public, the best way to achieve this is to make it
+free software which everyone can redistribute and change under these
+terms.
+
+To do so, attach the following notices to the program. It is safest to
+attach them to the start of each source file to most effectively state
+the exclusion of warranty; and each file should have at least the
+"copyright" line and a pointer to where the full notice is found.
+
+        <one line to give the program's name and a brief idea of what it does.>
+        Copyright (C) <year>  <name of author>
+
+        This program is free software: you can redistribute it and/or modify
+        it under the terms of the GNU General Public License as published by
+        the Free Software Foundation, either version 3 of the License, or
+        (at your option) any later version.
+
+        This program is distributed in the hope that it will be useful,
+        but WITHOUT ANY WARRANTY; without even the implied warranty of
+        MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+        GNU General Public License for more details.
+
+        You should have received a copy of the GNU General Public License
+        along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper
+mail.
+
+If the program does terminal interaction, make it output a short
+notice like this when it starts in an interactive mode:
+
+        <program>  Copyright (C) <year>  <name of author>
+        This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+        This is free software, and you are welcome to redistribute it
+        under certain conditions; type `show c' for details.
+
+The hypothetical commands \`show w' and \`show c' should show the
+appropriate parts of the General Public License. Of course, your
+program's commands might be different; for a GUI interface, you would
+use an "about box".
+
+You should also get your employer (if you work as a programmer) or
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. For more information on this, and how to apply and follow
+the GNU GPL, see <https://www.gnu.org/licenses/>.
+
+The GNU General Public License does not permit incorporating your
+program into proprietary programs. If your program is a subroutine
+library, you may consider it more useful to permit linking proprietary
+applications with the library. If this is what you want to do, use the
+GNU Lesser General Public License instead of this License. But first,
+please read <https://www.gnu.org/licenses/why-not-lgpl.html>.

LICENSES/GPL-3.0-or-later.txt

@@ -0,0 +1,232 @@
+GNU GENERAL PUBLIC LICENSE
+Version 3, 29 June 2007
+
+Copyright © 2007 Free Software Foundation, Inc. <https://fsf.org/>
+
+Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
+
+Preamble
+
+The GNU General Public License is a free, copyleft license for software and other kinds of works.
+
+The licenses for most software and other practical works are designed to take away your freedom to share and change the works. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change all versions of a program--to make sure it remains free software for all its users. We, the Free Software Foundation, use the GNU General Public License for most of our software; it applies also to any other work released this way by its authors. You can apply it to your programs, too.
+
+When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for them if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs, and that you know you can do these things.
+
+To protect your rights, we need to prevent others from denying you these rights or asking you to surrender the rights. Therefore, you have certain responsibilities if you distribute copies of the software, or if you modify it: responsibilities to respect the freedom of others.
+
+For example, if you distribute copies of such a program, whether gratis or for a fee, you must pass on to the recipients the same freedoms that you received. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.
+
+Developers that use the GNU GPL protect your rights with two steps: (1) assert copyright on the software, and (2) offer you this License giving you legal permission to copy, distribute and/or modify it.
+
+For the developers' and authors' protection, the GPL clearly explains that there is no warranty for this free software. For both users' and authors' sake, the GPL requires that modified versions be marked as changed, so that their problems will not be attributed erroneously to authors of previous versions.
+
+Some devices are designed to deny users access to install or run modified versions of the software inside them, although the manufacturer can do so. This is fundamentally incompatible with the aim of protecting users' freedom to change the software. The systematic pattern of such abuse occurs in the area of products for individuals to use, which is precisely where it is most unacceptable. Therefore, we have designed this version of the GPL to prohibit the practice for those products. If such problems arise substantially in other domains, we stand ready to extend this provision to those domains in future versions of the GPL, as needed to protect the freedom of users.
+
+Finally, every program is threatened constantly by software patents. States should not allow patents to restrict development and use of software on general-purpose computers, but in those that do, we wish to avoid the special danger that patents applied to a free program could make it effectively proprietary. To prevent this, the GPL assures that patents cannot be used to render the program non-free.
+
+The precise terms and conditions for copying, distribution and modification follow.
+
+TERMS AND CONDITIONS
+
+0. Definitions.
+
+“This License” refers to version 3 of the GNU General Public License.
+
+“Copyright” also means copyright-like laws that apply to other kinds of works, such as semiconductor masks.
+
+“The Program” refers to any copyrightable work licensed under this License. Each licensee is addressed as “you”. “Licensees” and “recipients” may be individuals or organizations.
+
+To “modify” a work means to copy from or adapt all or part of the work in a fashion requiring copyright permission, other than the making of an exact copy. The resulting work is called a “modified version” of the earlier work or a work “based on” the earlier work.
+
+A “covered work” means either the unmodified Program or a work based on the Program.
+
+To “propagate” a work means to do anything with it that, without permission, would make you directly or secondarily liable for infringement under applicable copyright law, except executing it on a computer or modifying a private copy. Propagation includes copying, distribution (with or without modification), making available to the public, and in some countries other activities as well.
+
+To “convey” a work means any kind of propagation that enables other parties to make or receive copies. Mere interaction with a user through a computer network, with no transfer of a copy, is not conveying.
+
+An interactive user interface displays “Appropriate Legal Notices” to the extent that it includes a convenient and prominently visible feature that (1) displays an appropriate copyright notice, and (2) tells the user that there is no warranty for the work (except to the extent that warranties are provided), that licensees may convey the work under this License, and how to view a copy of this License. If the interface presents a list of user commands or options, such as a menu, a prominent item in the list meets this criterion.
+
+1. Source Code.
+The “source code” for a work means the preferred form of the work for making modifications to it. “Object code” means any non-source form of a work.
+
+A “Standard Interface” means an interface that either is an official standard defined by a recognized standards body, or, in the case of interfaces specified for a particular programming language, one that is widely used among developers working in that language.
+
+The “System Libraries” of an executable work include anything, other than the work as a whole, that (a) is included in the normal form of packaging a Major Component, but which is not part of that Major Component, and (b) serves only to enable use of the work with that Major Component, or to implement a Standard Interface for which an implementation is available to the public in source code form. A “Major Component”, in this context, means a major essential component (kernel, window system, and so on) of the specific operating system (if any) on which the executable work runs, or a compiler used to produce the work, or an object code interpreter used to run it.
+
+The “Corresponding Source” for a work in object code form means all the source code needed to generate, install, and (for an executable work) run the object code and to modify the work, including scripts to control those activities. However, it does not include the work's System Libraries, or general-purpose tools or generally available free programs which are used unmodified in performing those activities but which are not part of the work. For example, Corresponding Source includes interface definition files associated with source files for the work, and the source code for shared libraries and dynamically linked subprograms that the work is specifically designed to require, such as by intimate data communication or control flow between those subprograms and other parts of the work.
+
+The Corresponding Source need not include anything that users can regenerate automatically from other parts of the Corresponding Source.
+
+The Corresponding Source for a work in source code form is that same work.
+
+2. Basic Permissions.
+All rights granted under this License are granted for the term of copyright on the Program, and are irrevocable provided the stated conditions are met. This License explicitly affirms your unlimited permission to run the unmodified Program. The output from running a covered work is covered by this License only if the output, given its content, constitutes a covered work. This License acknowledges your rights of fair use or other equivalent, as provided by copyright law.
+
+You may make, run and propagate covered works that you do not convey, without conditions so long as your license otherwise remains in force. You may convey covered works to others for the sole purpose of having them make modifications exclusively for you, or provide you with facilities for running those works, provided that you comply with the terms of this License in conveying all material for which you do not control copyright. Those thus making or running the covered works for you must do so exclusively on your behalf, under your direction and control, on terms that prohibit them from making any copies of your copyrighted material outside their relationship with you.
+
+Conveying under any other circumstances is permitted solely under the conditions stated below. Sublicensing is not allowed; section 10 makes it unnecessary.
+
+3. Protecting Users' Legal Rights From Anti-Circumvention Law.
+No covered work shall be deemed part of an effective technological measure under any applicable law fulfilling obligations under article 11 of the WIPO copyright treaty adopted on 20 December 1996, or similar laws prohibiting or restricting circumvention of such measures.
+
+When you convey a covered work, you waive any legal power to forbid circumvention of technological measures to the extent such circumvention is effected by exercising rights under this License with respect to the covered work, and you disclaim any intention to limit operation or modification of the work as a means of enforcing, against the work's users, your or third parties' legal rights to forbid circumvention of technological measures.
+
+4. Conveying Verbatim Copies.
+You may convey verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice; keep intact all notices stating that this License and any non-permissive terms added in accord with section 7 apply to the code; keep intact all notices of the absence of any warranty; and give all recipients a copy of this License along with the Program.
+
+You may charge any price or no price for each copy that you convey, and you may offer support or warranty protection for a fee.
+
+5. Conveying Modified Source Versions.
+You may convey a work based on the Program, or the modifications to produce it from the Program, in the form of source code under the terms of section 4, provided that you also meet all of these conditions:
+
+     a) The work must carry prominent notices stating that you modified it, and giving a relevant date.
+
+     b) The work must carry prominent notices stating that it is released under this License and any conditions added under section 7. This requirement modifies the requirement in section 4 to “keep intact all notices”.
+
+     c) You must license the entire work, as a whole, under this License to anyone who comes into possession of a copy. This License will therefore apply, along with any applicable section 7 additional terms, to the whole of the work, and all its parts, regardless of how they are packaged. This License gives no permission to license the work in any other way, but it does not invalidate such permission if you have separately received it.
+
+     d) If the work has interactive user interfaces, each must display Appropriate Legal Notices; however, if the Program has interactive interfaces that do not display Appropriate Legal Notices, your work need not make them do so.
+
+A compilation of a covered work with other separate and independent works, which are not by their nature extensions of the covered work, and which are not combined with it such as to form a larger program, in or on a volume of a storage or distribution medium, is called an “aggregate” if the compilation and its resulting copyright are not used to limit the access or legal rights of the compilation's users beyond what the individual works permit. Inclusion of a covered work in an aggregate does not cause this License to apply to the other parts of the aggregate.
+
+6. Conveying Non-Source Forms.
+You may convey a covered work in object code form under the terms of sections 4 and 5, provided that you also convey the machine-readable Corresponding Source under the terms of this License, in one of these ways:
+
+     a) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by the Corresponding Source fixed on a durable physical medium customarily used for software interchange.
+
+     b) Convey the object code in, or embodied in, a physical product (including a physical distribution medium), accompanied by a written offer, valid for at least three years and valid for as long as you offer spare parts or customer support for that product model, to give anyone who possesses the object code either (1) a copy of the Corresponding Source for all the software in the product that is covered by this License, on a durable physical medium customarily used for software interchange, for a price no more than your reasonable cost of physically performing this conveying of source, or (2) access to copy the Corresponding Source from a network server at no charge.
+
+     c) Convey individual copies of the object code with a copy of the written offer to provide the Corresponding Source. This alternative is allowed only occasionally and noncommercially, and only if you received the object code with such an offer, in accord with subsection 6b.
+
+     d) Convey the object code by offering access from a designated place (gratis or for a charge), and offer equivalent access to the Corresponding Source in the same way through the same place at no further charge. You need not require recipients to copy the Corresponding Source along with the object code. If the place to copy the object code is a network server, the Corresponding Source may be on a different server (operated by you or a third party) that supports equivalent copying facilities, provided you maintain clear directions next to the object code saying where to find the Corresponding Source. Regardless of what server hosts the Corresponding Source, you remain obligated to ensure that it is available for as long as needed to satisfy these requirements.
+
+     e) Convey the object code using peer-to-peer transmission, provided you inform other peers where the object code and Corresponding Source of the work are being offered to the general public at no charge under subsection 6d.
+
+A separable portion of the object code, whose source code is excluded from the Corresponding Source as a System Library, need not be included in conveying the object code work.
+
+A “User Product” is either (1) a “consumer product”, which means any tangible personal property which is normally used for personal, family, or household purposes, or (2) anything designed or sold for incorporation into a dwelling. In determining whether a product is a consumer product, doubtful cases shall be resolved in favor of coverage. For a particular product received by a particular user, “normally used” refers to a typical or common use of that class of product, regardless of the status of the particular user or of the way in which the particular user actually uses, or expects or is expected to use, the product. A product is a consumer product regardless of whether the product has substantial commercial, industrial or non-consumer uses, unless such uses represent the only significant mode of use of the product.
+
+“Installation Information” for a User Product means any methods, procedures, authorization keys, or other information required to install and execute modified versions of a covered work in that User Product from a modified version of its Corresponding Source. The information must suffice to ensure that the continued functioning of the modified object code is in no case prevented or interfered with solely because modification has been made.
+
+If you convey an object code work under this section in, or with, or specifically for use in, a User Product, and the conveying occurs as part of a transaction in which the right of possession and use of the User Product is transferred to the recipient in perpetuity or for a fixed term (regardless of how the transaction is characterized), the Corresponding Source conveyed under this section must be accompanied by the Installation Information. But this requirement does not apply if neither you nor any third party retains the ability to install modified object code on the User Product (for example, the work has been installed in ROM).
+
+The requirement to provide Installation Information does not include a requirement to continue to provide support service, warranty, or updates for a work that has been modified or installed by the recipient, or for the User Product in which it has been modified or installed. Access to a network may be denied when the modification itself materially and adversely affects the operation of the network or violates the rules and protocols for communication across the network.
+
+Corresponding Source conveyed, and Installation Information provided, in accord with this section must be in a format that is publicly documented (and with an implementation available to the public in source code form), and must require no special password or key for unpacking, reading or copying.
+
+7. Additional Terms.
+“Additional permissions” are terms that supplement the terms of this License by making exceptions from one or more of its conditions. Additional permissions that are applicable to the entire Program shall be treated as though they were included in this License, to the extent that they are valid under applicable law. If additional permissions apply only to part of the Program, that part may be used separately under those permissions, but the entire Program remains governed by this License without regard to the additional permissions.
+
+When you convey a copy of a covered work, you may at your option remove any additional permissions from that copy, or from any part of it. (Additional permissions may be written to require their own removal in certain cases when you modify the work.) You may place additional permissions on material, added by you to a covered work, for which you have or can give appropriate copyright permission.
+
+Notwithstanding any other provision of this License, for material you add to a covered work, you may (if authorized by the copyright holders of that material) supplement the terms of this License with terms:
+
+     a) Disclaiming warranty or limiting liability differently from the terms of sections 15 and 16 of this License; or
+
+     b) Requiring preservation of specified reasonable legal notices or author attributions in that material or in the Appropriate Legal Notices displayed by works containing it; or
+
+     c) Prohibiting misrepresentation of the origin of that material, or requiring that modified versions of such material be marked in reasonable ways as different from the original version; or
+
+     d) Limiting the use for publicity purposes of names of licensors or authors of the material; or
+
+     e) Declining to grant rights under trademark law for use of some trade names, trademarks, or service marks; or
+
+     f) Requiring indemnification of licensors and authors of that material by anyone who conveys the material (or modified versions of it) with contractual assumptions of liability to the recipient, for any liability that these contractual assumptions directly impose on those licensors and authors.
+
+All other non-permissive additional terms are considered “further restrictions” within the meaning of section 10. If the Program as you received it, or any part of it, contains a notice stating that it is governed by this License along with a term that is a further restriction, you may remove that term. If a license document contains a further restriction but permits relicensing or conveying under this License, you may add to a covered work material governed by the terms of that license document, provided that the further restriction does not survive such relicensing or conveying.
+
+If you add terms to a covered work in accord with this section, you must place, in the relevant source files, a statement of the additional terms that apply to those files, or a notice indicating where to find the applicable terms.
+
+Additional terms, permissive or non-permissive, may be stated in the form of a separately written license, or stated as exceptions; the above requirements apply either way.
+
+8. Termination.
+You may not propagate or modify a covered work except as expressly provided under this License. Any attempt otherwise to propagate or modify it is void, and will automatically terminate your rights under this License (including any patent licenses granted under the third paragraph of section 11).
+
+However, if you cease all violation of this License, then your license from a particular copyright holder is reinstated (a) provisionally, unless and until the copyright holder explicitly and finally terminates your license, and (b) permanently, if the copyright holder fails to notify you of the violation by some reasonable means prior to 60 days after the cessation.
+
+Moreover, your license from a particular copyright holder is reinstated permanently if the copyright holder notifies you of the violation by some reasonable means, this is the first time you have received notice of violation of this License (for any work) from that copyright holder, and you cure the violation prior to 30 days after your receipt of the notice.
+
+Termination of your rights under this section does not terminate the licenses of parties who have received copies or rights from you under this License. If your rights have been terminated and not permanently reinstated, you do not qualify to receive new licenses for the same material under section 10.
+
+9. Acceptance Not Required for Having Copies.
+You are not required to accept this License in order to receive or run a copy of the Program. Ancillary propagation of a covered work occurring solely as a consequence of using peer-to-peer transmission to receive a copy likewise does not require acceptance. However, nothing other than this License grants you permission to propagate or modify any covered work. These actions infringe copyright if you do not accept this License. Therefore, by modifying or propagating a covered work, you indicate your acceptance of this License to do so.
+
+10. Automatic Licensing of Downstream Recipients.
+Each time you convey a covered work, the recipient automatically receives a license from the original licensors, to run, modify and propagate that work, subject to this License. You are not responsible for enforcing compliance by third parties with this License.
+
+An “entity transaction” is a transaction transferring control of an organization, or substantially all assets of one, or subdividing an organization, or merging organizations. If propagation of a covered work results from an entity transaction, each party to that transaction who receives a copy of the work also receives whatever licenses to the work the party's predecessor in interest had or could give under the previous paragraph, plus a right to possession of the Corresponding Source of the work from the predecessor in interest, if the predecessor has it or can get it with reasonable efforts.
+
+You may not impose any further restrictions on the exercise of the rights granted or affirmed under this License. For example, you may not impose a license fee, royalty, or other charge for exercise of rights granted under this License, and you may not initiate litigation (including a cross-claim or counterclaim in a lawsuit) alleging that any patent claim is infringed by making, using, selling, offering for sale, or importing the Program or any portion of it.
+
+11. Patents.
+A “contributor” is a copyright holder who authorizes use under this License of the Program or a work on which the Program is based. The work thus licensed is called the contributor's “contributor version”.
+
+A contributor's “essential patent claims” are all patent claims owned or controlled by the contributor, whether already acquired or hereafter acquired, that would be infringed by some manner, permitted by this License, of making, using, or selling its contributor version, but do not include claims that would be infringed only as a consequence of further modification of the contributor version. For purposes of this definition, “control” includes the right to grant patent sublicenses in a manner consistent with the requirements of this License.
+
+Each contributor grants you a non-exclusive, worldwide, royalty-free patent license under the contributor's essential patent claims, to make, use, sell, offer for sale, import and otherwise run, modify and propagate the contents of its contributor version.
+
+In the following three paragraphs, a “patent license” is any express agreement or commitment, however denominated, not to enforce a patent (such as an express permission to practice a patent or covenant not to sue for patent infringement). To “grant” such a patent license to a party means to make such an agreement or commitment not to enforce a patent against the party.
+
+If you convey a covered work, knowingly relying on a patent license, and the Corresponding Source of the work is not available for anyone to copy, free of charge and under the terms of this License, through a publicly available network server or other readily accessible means, then you must either (1) cause the Corresponding Source to be so available, or (2) arrange to deprive yourself of the benefit of the patent license for this particular work, or (3) arrange, in a manner consistent with the requirements of this License, to extend the patent license to downstream recipients. “Knowingly relying” means you have actual knowledge that, but for the patent license, your conveying the covered work in a country, or your recipient's use of the covered work in a country, would infringe one or more identifiable patents in that country that you have reason to believe are valid.
+
+If, pursuant to or in connection with a single transaction or arrangement, you convey, or propagate by procuring conveyance of, a covered work, and grant a patent license to some of the parties receiving the covered work authorizing them to use, propagate, modify or convey a specific copy of the covered work, then the patent license you grant is automatically extended to all recipients of the covered work and works based on it.
+
+A patent license is “discriminatory” if it does not include within the scope of its coverage, prohibits the exercise of, or is conditioned on the non-exercise of one or more of the rights that are specifically granted under this License. You may not convey a covered work if you are a party to an arrangement with a third party that is in the business of distributing software, under which you make payment to the third party based on the extent of your activity of conveying the work, and under which the third party grants, to any of the parties who would receive the covered work from you, a discriminatory patent license (a) in connection with copies of the covered work conveyed by you (or copies made from those copies), or (b) primarily for and in connection with specific products or compilations that contain the covered work, unless you entered into that arrangement, or that patent license was granted, prior to 28 March 2007.
+
+Nothing in this License shall be construed as excluding or limiting any implied license or other defenses to infringement that may otherwise be available to you under applicable patent law.
+
+12. No Surrender of Others' Freedom.
+If conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot convey a covered work so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not convey it at all. For example, if you agree to terms that obligate you to collect a royalty for further conveying from those to whom you convey the Program, the only way you could satisfy both those terms and this License would be to refrain entirely from conveying the Program.
+
+13. Use with the GNU Affero General Public License.
+Notwithstanding any other provision of this License, you have permission to link or combine any covered work with a work licensed under version 3 of the GNU Affero General Public License into a single combined work, and to convey the resulting work. The terms of this License will continue to apply to the part which is the covered work, but the special requirements of the GNU Affero General Public License, section 13, concerning interaction through a network will apply to the combination as such.
+
+14. Revised Versions of this License.
+The Free Software Foundation may publish revised and/or new versions of the GNU General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
+
+Each version is given a distinguishing version number. If the Program specifies that a certain numbered version of the GNU General Public License “or any later version” applies to it, you have the option of following the terms and conditions either of that numbered version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of the GNU General Public License, you may choose any version ever published by the Free Software Foundation.
+
+If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program.
+
+Later license versions may give you additional or different permissions. However, no additional obligations are imposed on any author or copyright holder as a result of your choosing to follow a later version.
+
+15. Disclaimer of Warranty.
+THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
+
+16. Limitation of Liability.
+IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
+
+17. Interpretation of Sections 15 and 16.
+If the disclaimer of warranty and limitation of liability provided above cannot be given local legal effect according to their terms, reviewing courts shall apply local law that most closely approximates an absolute waiver of all civil liability in connection with the Program, unless a warranty or assumption of liability accompanies a copy of the Program in return for a fee.
+
+END OF TERMS AND CONDITIONS
+
+How to Apply These Terms to Your New Programs
+
+If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
+
+To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively state the exclusion of warranty; and each file should have at least the “copyright” line and a pointer to where the full notice is found.
+
+     <one line to give the program's name and a brief idea of what it does.>
+     Copyright (C) <year>  <name of author>
+
+     This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
+
+     This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
+
+     You should have received a copy of the GNU General Public License along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+Also add information on how to contact you by electronic and paper mail.
+
+If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode:
+
+     <program>  Copyright (C) <year>  <name of author>
+     This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
+     This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details.
+
+The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, your program's commands might be different; for a GUI interface, you would use an “about box”.
+
+You should also get your employer (if you work as a programmer) or school, if any, to sign a “copyright disclaimer” for the program, if necessary. For more information on this, and how to apply and follow the GNU GPL, see <https://www.gnu.org/licenses/>.
+
+The GNU General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License. But first, please read <https://www.gnu.org/philosophy/why-not-lgpl.html>.

LICENSES/MIT.txt

@@ -0,0 +1,9 @@
+MIT License
+
+Copyright (c) <year> <copyright holders>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

MAINTAINERS.md

@@ -0,0 +1,10 @@
+# Maintainers
+
+This file contains information about people permitted to make major decisions and direction on the project.
+
+## Contributing Under the DCO
+
+By adding your name and email address to this section, you certify that all of your subsequent contributions to
+gp2040ce-binary-tools are made under the terms of the Developer's Certificate of Origin 1.1, available at `DCO.txt`.
+
+* Brian S. Stephan (<bss@incorporeal.org>)

README.md

@@ -4,65 +4,150 @@ Tools for working with GP2040-CE binary dumps.
 
 ## Dependencies
 
-Interacting with your board (e.g. getting dumps, etc.) requires [picotool](https://github.com/raspberrypi/picotool), and
-currently the expectation is that you can run it yourself before invoking these tools. That may change one day.
+While not necessary for most tools, you may want [picotool](https://github.com/raspberrypi/picotool) as an alternative
+way to dump binary data from the board. These dumps can be created with `gp2040ce-binary-tools` natively, but having an
+alternative way to create a binary dump can be helpful, as these tools work as well (or better) with a binary dump as
+over USB.
+
+### Protobuf Files
+
+All tools take `-P PATH` flag(s) in order to import Protobuf files (either precompiled Python files or raw .proto files)
+if you have them locally, in order to work with the latest (or development) version of the configuration. That said,
+this tool also includes a copy of the config files if you cannot supply these files, and will attempt to compile those
+as a fallback. Be aware, however, that they are a point in time snapshot, and may lag the real format in undesirable
+ways. Supply the latest Protobuf files if you can.
+
+An example of this invocation is:
+
+`visualize-config -P ~/proj/GP2040-CE/proto -P ~/proj/GP2040-CE/lib/nanopb/generator/proto --filename memory.bin`
 
 ## Installation
 
+```
+% pip install gp2040ce-binary-tools
+```
+
+### Development Installation
+
 ```
 % git clone [URL to this repository]
 % cd gp2040ce-binary-tools
 % python -m venv venv
 % source ./venv/bin/activate
 % pip install -e .
+% pip install -Ur requirements/requirements-dev.txt
 ```
 
-At some point we may publish packages to e.g. pypi.
-
-### Development Installation
-
-As above, plus also `pip install -Ur requirements/requirements-dev.txt` to get linters and whatnot.
-
 ## Tools
 
 In all cases, online help can be retrieved by providing the `-h` or ``--help`` flags to the below programs.
 
+### Config Editor
+
+[![asciicast](https://asciinema.org/a/67hELtUNkKCit4dFwYeAUa2fo.svg)](https://asciinema.org/a/67hELtUNkKCit4dFwYeAUa2fo)
+
+A terminal UI config editor, capable of viewing and editing existing configurations, can be launched via
+`edit-config`. It supports navigation both via the keyboard or the mouse, and can view and edit either a binary file
+made via `picotool` or configuration directly on the board in BOOTSEL mode over USB.
+
+Simple usage:
+
+| Key(s)                | Action                                                 |
+|-----------------------|--------------------------------------------------------|
+| Up, Down              | Move up and down the config tree                       |
+| Left, Right           | Scroll the tree left and right (when content is long)  |
+| Space                 | Expand a tree node to show its children                |
+| Enter                 | Expand a tree node, or edit a leaf node (bools toggle) |
+| Tab (in edit popup)   | Cycle highlight between input field and buttons        |
+| Enter (in edit popup) | Choose dropdown option or activate button              |
+| S                     | Save the config to the opened file                     |
+| Q                     | Quit without saving                                    |
+
+A quick demonstration of the editor is available [on asciinema.org](https://asciinema.org/a/67hELtUNkKCit4dFwYeAUa2fo).
+
 ### concatenate
 
-**concatenate** combines a GP2040-CE firmware .bin file (such as from a fresh build) and a GP2040-CE board's storage
-section .bin or config (with footer) .bin, and produces a properly-offset .bin file suitable for flashing to a board.
-This may be useful to ensure the board is flashed with a particular configuration, for instances such as producing a
-binary to flash many boards with a particular configuration (specific customizations, etc.), or keeping documented
-backups of what you're testing with during development.
+`concatenate` combines a GP2040-CE firmware .bin file (such as from a fresh build) with:
+
+* a GP2040-CE board config, in the form of
+    * a config section .bin (with footer) (optionally padded) (`--binary-board-config-filename`) or
+    * a JSON file representing the config (`--json-board-config-filename`)
+* and/or a GP2040-CE user config, in the form of
+    * a config section .bin (with footer) (optionally padded) (`--binary-user-config-filename`) or
+    * a JSON file representing the config (`--json-user-config-filename`)
+
+...and produces a properly-offset firmware file suitable for flashing to a board with the provided config(s). This may
+be useful to ensure the board is flashed with a particular configuration, for instances such as producing a binary to
+flash many boards with a particular configuration (specific customizations, etc.), creating a file suitable for the
+initial install of a fresh board (a "board config"), or keeping documented backups of what you're testing with during
+development.
+
+The `--...-board-config-filename` flags allow for shipping a default configuration as part of the binary, replacing
+the need for generating these board configurations at compile time. This allows for more custom builds and less
+dependency on the build jobs, and is a feature in progress in the core firmware.
+
+The produced firmware + config(s) can be written to a file with `--new-filename FILENAME` or straight to a RP2040
+in BOOTSEL mode with `--usb`. The output file is a direct binary representation by default, but if `FILENAME` ends in
+".uf2", it will be written in the UF2 format, which is generally more convenient to the end user.
 
 Sample usage:
 
 ```
-% concatenate build/GP2040-CE_foo_bar.bin storage-dump.bin new-firmware-with-config.bin
+% concatenate build/GP2040-CE_foo_bar.bin --binary-user-config-filename storage-dump.bin \
+    --new-filename new-firmware-with-config.bin
 ```
 
-### visualize-storage
+### dump-config
 
-**visualize-storage** reads a dump of a GP2040-CE board's flash storage section, where the configuration lives, and
-prints it out for visual inspection or diffing with other tools. It can also find the storage section from a GP2040-CE
-whole board dump, if you have that instead. Usage is simple; just pass the tool your binary file to analyze along with
-the path to the Protobuf files.
+`dump-config` replaces the need for picotool in order to make a copy of the GP2040-CE configuration as a binary file.
+This could be used with the other tools, or just to keep a backup.
 
-Because Protobuf relies on .proto files to convey the serialized structure, you must supply them from the main GP2040-CE
-project, e.g. pointing this tool at your clone of the core project. Something like this would suffice for a working
-invocation (note: you do not need to compile the files yourself):
+Sample usage:
 
 ```
-% visualize-storage -P ~/proj/GP2040-CE/proto -P ~/proj/GP2040-CE/lib/nanopb/generator/proto memory.bin
+% dump-config `date +%Y%m%d`-config-backup.bin
 ```
 
-(In the future we will look into publishing complete packages that include the compiled `_pb2.py` files, so that you
-don't need to provide them yourself.)
+### dump-gp2040ce
+
+`dump-gp2040ce` replaces the need for picotool in order to make a copy of a board's full GP2040-CE image as a binary file.
+This could be used with the other tools, or just to keep a backup.
+
+Sample usage:
+
+```
+% dump-gp2040ce `date +%Y%m%d`-backup.bin
+```
+
+### summarize-gp2040ce
+
+`summarize-gp2040ce` prints information regarding the provided USB device or file. It attempts to detect the firmware
+and/or board config and/or user config version, which might be useful for confirming files are built properly, or to
+determine the lineage of something.
+
+Sample usage:
+
+```
+% summarize-gp2040ce --usb
+USB device:
+
+GP2040-CE Information
+  detected GP2040-CE version:     v0.7.8-9-g59e2d19b-dirty
+  detected board config version:  v0.7.8-board-test
+  detected user config version:   v0.7.8-RC2-1-g882235b3
+```
+
+### visualize-config
+
+`visualize-config` reads a GP2040-CE board's configuration, either over USB or from a dump of the board's flash
+storage section, and prints it out for visual inspection or diffing with other tools. It can also find the storage
+section from a GP2040-CE whole board dump, if you have that instead. Usage is simple; just connect your board in BOOTSEL
+mode or pass the tool your binary file to analyze along with the path to the Protobuf files.
 
 Sample output:
 
 ```
-% visualize-storage -P ~/proj/GP2040-CE/proto -P ~/proj/GP2040-CE/lib/nanopb/generator/proto ~/proj/GP2040-CE/demo-memory.bin
+% visualize-config --usb
 boardVersion: "v0.7.2"
 gamepadOptions {
   inputMode: INPUT_MODE_HID
@@ -127,13 +212,31 @@ forcedSetupOptions {
 }
 ```
 
-### Dumping the GP2040-CE board
+## Miscellaneous
 
-These tools require a dump of your GP2040-CE board, either the storage section or the whole board, depending on the
-context. The storage section of a GP2040-CE board is a reserved 8 KB starting at `0x101FE000`. To dump your board's storage:
+### Version information
+
+The GP2040-CE configuration is still changing, so the tools are changing accordingly. This project doesn't currently make
+a huge effort to be backwards compatible, so instead, refer to this:
+
+#### Flash Layouts
+
+* `gp2040ce-binary-tools >=v0.6.0` supports both board and user configs still being developed in `GP2040-CE`.
+* `gp2040ce-binary-tools >=v0.5.1` supported the increased user config size in `GP2040-CE >=v0.7.5`.
+* `gp2040ce-binary-tools <=v0.5.0` supported the smaller user config size in `GP2040-CE <v0.7.5`.
+
+#### Config Structures
+
+The latest update of the configuration snapshot is from **v0.7.8**.
+
+### Dumping the GP2040-CE board with picotool
+
+Some of these tools require a dump of your GP2040-CE board, either the storage section or the whole board, depending on
+the context. The user config storage section of a GP2040-CE board is a reserved 32 KB starting at `0x101F8000`. To dump
+your board's storage with picotool:
 
 ```
-% picotool save -r 101FE000 10200000 memory.bin
+% picotool save -r 101F8000 10200000 memory.bin
 ```
 
 And to dump your whole board:
@@ -141,3 +244,18 @@ And to dump your whole board:
 ```
 % picotool save -a whole-board.bin
 ```
+
+## Author and Licensing
+
+Written by and copyright Brian S. Stephan (<bss@incorporeal.org>).
+
+gp2040ce-binary-tools is free software: you can redistribute it and/or modify it under the terms of the GNU General
+Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any
+later version.
+
+gp2040ce-binary-tools is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the
+implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
+details.
+
+You should have received a copy of the GNU General Public License along with gp2040ce-binary-tools. If not, see
+<https://www.gnu.org/licenses/>.

gp2040ce_bintools/__init__.py

@@ -1,4 +1,8 @@
-"""Initialize the package and get dependencies."""
+"""Initialize the package and get dependencies.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
 import argparse
 import importlib
 import logging
@@ -29,12 +33,15 @@ logger = logging.getLogger(__name__)
 
 # parse flags that are common to many tools (e.g. adding paths for finding .proto files)
 core_parser = argparse.ArgumentParser(add_help=False)
-core_parser.add_argument('-v', '--version', action='version', version=f"%(prog)s {__version__}",
+core_parser.add_argument('-v', '--version', action='version',
+                         version=f"gp2040ce-binary-tools {__version__} (Python {sys.version})",
                          help="show version information and exit")
 core_parser.add_argument('-d', '--debug', action='store_true', help="enable debug logging")
 core_parser.add_argument('-P', '--proto-files-path', type=pathlib.Path, default=list(), action='append',
                          help="path to .proto files to read, including dependencies; you will likely need "
                               "to supply this twice, once for GP2040-CE's .proto files and once for nanopb's")
+core_parser.add_argument('-S', '--use-shipped-fallback', action='store_true',
+                         help="utilize shipped (potentially stale) .proto files because you can't supply your own")
 args, _ = core_parser.parse_known_args()
 for path in args.proto_files_path:
     sys.path.append(os.path.abspath(os.path.expanduser(path)))
@@ -45,11 +52,27 @@ else:
     handler.setLevel(logging.WARNING)
 
 
-def get_config_pb2():
+def get_config_pb2(with_fallback: bool = args.use_shipped_fallback):
     """Retrieve prebuilt _pb2 file or attempt to compile it live."""
+    # try to just import a precompiled module if we have been given it in our path
+    # (perhaps someone already compiled it for us for whatever reason)
     try:
+        logger.debug("Trying precompiled protobuf modules...")
         return importlib.import_module('config_pb2')
     except ModuleNotFoundError:
-        # compile the proto files in realtime, leave them in this package
-        logger.info("Invoking gRPC tool to compile config.proto...")
-        return grpc.protos('config.proto')
+        # no found precompiled config, try to compile the proto files in realtime
+        # because it's possible someone put them on the path
+        try:
+            logger.debug("No precompiled protobuf modules found, invoking gRPC tool to compile config.proto...")
+            return grpc.protos('config.proto')
+        except (ModuleNotFoundError, TypeError):
+            # (TypeError could be the windows bug https://github.com/protocolbuffers/protobuf/issues/14345)
+            if not with_fallback:
+                logger.exception("no viable set of protobuf modules could be found, please use -P or -S!")
+                raise RuntimeError("no viable set of protobuf modules could be found, please use -P or -S!")
+
+            # that failed, import the snapshot (may be lagging what's in GP2040-CE)
+            logger.warning("using the fallback .proto files! please supply your files with -P if you can!")
+            sys.path.append(os.path.join(pathlib.Path(__file__).parent.resolve(), 'proto_snapshot'))
+            logger.debug("Invoking gRPC tool again to compile shipped config.proto...")
+            return grpc.protos('config.proto')

gp2040ce_bintools/builder.py

@@ -1,12 +1,27 @@
-"""Build binary files for a GP2040-CE board."""
-import argparse
-import logging
+"""Build binary files for a GP2040-CE board.
 
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
+import argparse
+import copy
+import logging
+import os
+import re
+from typing import Optional
+
+from google.protobuf.json_format import MessageToJson
+from google.protobuf.message import Message
+
+import gp2040ce_bintools.storage as storage
 from gp2040ce_bintools import core_parser
-from gp2040ce_bintools.storage import STORAGE_LOCATION, pad_config_to_storage_size
+from gp2040ce_bintools.rp2040 import get_bootsel_endpoints, read, write
 
 logger = logging.getLogger(__name__)
 
+GP2040CE_START_ADDRESS = 0x10000000
+GP2040CE_SIZE = 2 * 1024 * 1024
+
 
 #################
 # LIBRARY ITEMS #
@@ -17,65 +32,350 @@ class FirmwareLengthError(ValueError):
     """Exception raised when the firmware is too large to fit the known storage location."""
 
 
-def combine_firmware_and_config(firmware_binary: bytearray, config_binary: bytearray) -> bytearray:
-    """Given firmware and config binaries, combine the two to one, with proper offsets for GP2040-CE.
+def combine_firmware_and_config(firmware_binary: bytearray, board_config_binary: bytearray,
+                                user_config_binary: bytearray, replace_extra: bool = False) -> bytearray:
+    """Given firmware and board and/or user config binaries, combine to one binary with proper offsets for GP2040-CE.
 
     Args:
         firmware_binary: binary data of the raw GP2040-CE firmware, probably but not necessarily unpadded
-        config_binary: binary data of board config + footer, possibly padded to be a full storage section
+        board_config_binary: binary data of board config + footer, possibly padded to be a full storage section
+        user_config_binary: binary data of user config + footer, possibly padded to be a full storage section
+        replace_extra: if larger than normal firmware files should have their overage replaced
     Returns:
         the resulting correctly-offset binary suitable for a GP2040-CE board
     """
-    return pad_firmware_up_to_storage(firmware_binary) + pad_config_to_storage_size(config_binary)
+    if not board_config_binary and not user_config_binary:
+        raise ValueError("at least one config binary must be provided!")
+
+    combined = copy.copy(firmware_binary)
+    if board_config_binary:
+        combined = (pad_binary_up_to_board_config(combined, or_truncate=replace_extra) +
+                    storage.pad_config_to_storage_size(board_config_binary))
+    if user_config_binary:
+        combined = (pad_binary_up_to_user_config(combined, or_truncate=replace_extra) +
+                    storage.pad_config_to_storage_size(user_config_binary))
+    return combined
 
 
-def concatenate_firmware_and_storage_files(firmware_filename: str, storage_filename: str, combined_filename: str):
+def concatenate_firmware_and_storage_files(firmware_filename: str,      # noqa: C901
+                                           binary_board_config_filename: Optional[str] = None,
+                                           json_board_config_filename: Optional[str] = None,
+                                           binary_user_config_filename: Optional[str] = None,
+                                           json_user_config_filename: Optional[str] = None,
+                                           combined_filename: str = '', usb: bool = False,
+                                           replace_extra: bool = False,
+                                           backup: bool = False) -> None:
     """Open the provided binary files and combine them into one combined GP2040-CE with config file.
 
     Args:
         firmware_filename: filename of the firmware binary to read
-        storage_filename: filename of the storage section to read
+        binary_board_config_filename: filename of the board config section to read, in binary format
+        json_board_config_filename: filename of the board config section to read, in JSON format
+        binary_user_config_filename: filename of the user config section to read, in binary format
+        json_user_config_filename: filename of the user config section to read, in JSON format
         combined_filename: filename of where to write the combine binary
+        replace_extra: if larger than normal firmware files should have their overage replaced
+        backup: if the output filename exists, move it to foo.ext.old before writing foo.ext
     """
-    with open(firmware_filename, 'rb') as firmware, open(storage_filename, 'rb') as storage:
-        new_binary = combine_firmware_and_config(bytearray(firmware.read()), bytearray(storage.read()))
-    with open(combined_filename, 'wb') as combined:
-        combined.write(new_binary)
+    new_binary = bytearray([])
+    board_config_binary = bytearray([])
+    user_config_binary = bytearray([])
+
+    if binary_board_config_filename:
+        with open(binary_board_config_filename, 'rb') as binary_file:
+            board_config_binary = bytearray(binary_file.read())
+    elif json_board_config_filename:
+        with open(json_board_config_filename, 'r') as json_file:
+            config = storage.get_config_from_json(json_file.read())
+            board_config_binary = storage.serialize_config_with_footer(config)
+
+    if binary_user_config_filename:
+        with open(binary_user_config_filename, 'rb') as binary_file:
+            user_config_binary = bytearray(binary_file.read())
+    elif json_user_config_filename:
+        with open(json_user_config_filename, 'r') as json_file:
+            config = storage.get_config_from_json(json_file.read())
+            user_config_binary = storage.serialize_config_with_footer(config)
+
+    with open(firmware_filename, 'rb') as firmware:
+        firmware_binary = bytearray(firmware.read())
+
+    # create a sequential binary for .bin and USB uses, or index it for .uf2
+    if usb or combined_filename[-4:] != '.uf2':
+        new_binary = combine_firmware_and_config(firmware_binary, board_config_binary, user_config_binary,
+                                                 replace_extra=replace_extra)
+    else:
+        binary_list = [(0, firmware_binary)]
+        # we must pad to storage start in order for the UF2 write addresses to make sense
+        if board_config_binary:
+            binary_list.append((storage.BOARD_CONFIG_BINARY_LOCATION,
+                                storage.pad_config_to_storage_size(board_config_binary)))
+        if user_config_binary:
+            binary_list.append((storage.USER_CONFIG_BINARY_LOCATION,
+                                storage.pad_config_to_storage_size(user_config_binary)))
+        new_binary = storage.convert_binary_to_uf2(binary_list)
+
+    if combined_filename:
+        if backup and os.path.exists(combined_filename):
+            os.rename(combined_filename, f'{combined_filename}.old')
+        with open(combined_filename, 'wb') as combined:
+            combined.write(new_binary)
+
+    if usb:
+        endpoint_out, endpoint_in = get_bootsel_endpoints()
+        write(endpoint_out, endpoint_in, GP2040CE_START_ADDRESS, bytes(new_binary))
 
 
-def pad_firmware_up_to_storage(firmware: bytes) -> bytearray:
+def find_version_string_in_binary(binary: bytes) -> str:
+    """Search for a git describe style version string in a binary file.
+
+    Args:
+        binary: the binary to search
+    Returns:
+        the first found string, or None
+    """
+    match = re.search(b'v[0-9]+.[0-9]+.[0-9]+[A-Za-z0-9-+.]*', binary)
+    if match:
+        return match.group(0).decode(encoding='ascii')
+    return 'NONE'
+
+
+def get_gp2040ce_from_usb() -> tuple[bytes, object, object]:
+    """Read the firmware + config sections from a USB device.
+
+    Returns:
+        the bytes from the board, along with the USB out and in endpoints for reference
+    """
+    # open the USB device and get the config
+    endpoint_out, endpoint_in = get_bootsel_endpoints()
+    logger.debug("reading DEVICE ID %s:%s, bus %s, address %s", hex(endpoint_out.device.idVendor),
+                 hex(endpoint_out.device.idProduct), endpoint_out.device.bus, endpoint_out.device.address)
+    content = read(endpoint_out, endpoint_in, GP2040CE_START_ADDRESS, GP2040CE_SIZE)
+    return content, endpoint_out, endpoint_in
+
+
+def pad_binary_up_to_address(binary: bytes, position: int, or_truncate: bool = False) -> bytearray:
     """Provide a copy of the firmware padded with zero bytes up to the provided position.
 
     Args:
-        firmware: the firmware binary to process
+        binary: the binary to process
+        position: the byte to pad to
+        or_truncate: if the firmware is longer than expected, just return the max size
     Returns:
         the resulting padded binary as a bytearray
     Raises:
         FirmwareLengthError: if the firmware is larger than the storage location
     """
-    bytes_to_pad = STORAGE_LOCATION - len(firmware)
-    logger.debug("firmware is length %s, padding %s bytes", len(firmware), bytes_to_pad)
+    bytes_to_pad = position - len(binary)
+    logger.debug("firmware is length %s, padding %s bytes", len(binary), bytes_to_pad)
     if bytes_to_pad < 0:
+        if or_truncate:
+            return bytearray(binary[0:position])
         raise FirmwareLengthError(f"provided firmware binary is larger than the start of "
-                                  f"storage at {STORAGE_LOCATION}!")
+                                  f"storage at {position}!")
 
-    return bytearray(firmware) + bytearray(b'\x00' * bytes_to_pad)
+    return bytearray(binary) + bytearray(b'\x00' * bytes_to_pad)
+
+
+def pad_binary_up_to_board_config(firmware: bytes, or_truncate: bool = False) -> bytearray:
+    """Provide a copy of the firmware padded with zero bytes up to the board config position.
+
+    Args:
+        firmware: the firmware binary to process
+        or_truncate: if the firmware is longer than expected, just return the max size
+    Returns:
+        the resulting padded binary as a bytearray
+    Raises:
+        FirmwareLengthError: if the firmware is larger than the storage location
+    """
+    return pad_binary_up_to_address(firmware, storage.BOARD_CONFIG_BINARY_LOCATION, or_truncate)
+
+
+def pad_binary_up_to_user_config(firmware: bytes, or_truncate: bool = False) -> bytearray:
+    """Provide a copy of the firmware padded with zero bytes up to the user config position.
+
+    Args:
+        firmware: the firmware binary to process
+        or_truncate: if the firmware is longer than expected, just return the max size
+    Returns:
+        the resulting padded binary as a bytearray
+    Raises:
+        FirmwareLengthError: if the firmware is larger than the storage location
+    """
+    return pad_binary_up_to_address(firmware, storage.USER_CONFIG_BINARY_LOCATION, or_truncate)
+
+
+def replace_config_in_binary(board_binary: bytearray, config_binary: bytearray) -> bytearray:
+    """Given (presumed) whole board and config binaries, combine the two to one, with proper offsets for GP2040-CE.
+
+    Whatever is in the board binary is not sanity checked, and is overwritten. If it is
+    too small to be a board dump, it is nonetheless expanded and overwritten to fit the
+    proper size.
+
+    Args:
+        board_binary: binary data of a whole board dump from a GP2040-CE board
+        config_binary: binary data of board config + footer, possibly padded to be a full storage section
+    Returns:
+        the resulting correctly-offset binary suitable for a GP2040-CE board
+    """
+    if len(board_binary) < storage.USER_CONFIG_BINARY_LOCATION + storage.STORAGE_SIZE:
+        # this is functionally the same, since this doesn't sanity check the firmware
+        return combine_firmware_and_config(board_binary, bytearray([]), config_binary)
+    else:
+        new_binary = bytearray(copy.copy(board_binary))
+        new_config = storage.pad_config_to_storage_size(config_binary)
+        location_end = storage.USER_CONFIG_BINARY_LOCATION + storage.STORAGE_SIZE
+        new_binary[storage.USER_CONFIG_BINARY_LOCATION:location_end] = new_config
+        return new_binary
+
+
+def write_new_config_to_filename(config: Message, filename: str, inject: bool = False) -> None:
+    """Serialize the provided config to the specified file.
+
+    The file may be replaced, creating a configuration section-only binary, or appended to
+    an existing file that is grown to place the config section in the proper location.
+
+    Args:
+        config: the Protobuf configuration to write to disk
+        filename: the filename to write the serialized configuration to
+        inject: if True, the file is read in and has its storage section replaced; if False,
+                the whole file is replaced
+    """
+    if inject:
+        config_binary = storage.serialize_config_with_footer(config)
+        with open(filename, 'rb') as file:
+            existing_binary = file.read()
+        binary = replace_config_in_binary(bytearray(existing_binary), config_binary)
+        with open(filename, 'wb') as file:
+            file.write(binary)
+    else:
+        if filename[-5:] == '.json':
+            with open(filename, 'w') as file:
+                file.write(f'{MessageToJson(config)}\n')
+        else:
+            binary = storage.serialize_config_with_footer(config)
+            with open(filename, 'wb') as file:
+                if filename[-4:] == '.uf2':
+                    # we must pad to storage start in order for the UF2 write addresses to make sense
+                    file.write(storage.convert_binary_to_uf2([
+                        (storage.USER_CONFIG_BINARY_LOCATION, storage.pad_config_to_storage_size(binary)),
+                    ]))
+                else:
+                    file.write(binary)
+
+
+def write_new_config_to_usb(config: Message, endpoint_out: object, endpoint_in: object):
+    """Serialize the provided config to a device over USB, in the proper location for a GP2040-CE board.
+
+    Args:
+        config: the Protobuf configuration to write to a RP2040 board in BOOTSEL mode
+        endpoint_out: the USB endpoint to write to
+        endpoint_in: the USB endpoint to read from
+    """
+    serialized = storage.serialize_config_with_footer(config)
+    # we don't write the whole area, just the minimum from the end of the storage section
+    # nevertheless, the USB device needs writes to start at 256 byte boundaries
+    logger.debug("serialized: %s", serialized)
+    # not sure why this minimal padding isn't working but it leads to corruption
+    # maybe claims that erase need to be on 4096 byte sectors?
+    # padding = 256 - (len(serialized) % 256)
+    padding = 4096 - (len(serialized) % 4096)
+    logger.debug("length: %s with %s bytes of padding", len(serialized), padding)
+    binary = bytearray(b'\x00' * padding) + serialized
+    logger.debug("binary for writing: %s", binary)
+    write(endpoint_out, endpoint_in, storage.USER_CONFIG_BOOTSEL_ADDRESS + (storage.STORAGE_SIZE - len(binary)),
+          bytes(binary))
 
 
 ############
 # COMMANDS #
 ############
 
+
 def concatenate():
     """Combine a built firmware .bin and a storage .bin."""
     parser = argparse.ArgumentParser(
-        description="Combine a compiled GP2040-CE firmware-only .bin and an existing storage area or config .bin "
-                    "into one file suitable for flashing onto a board.",
+        description="Combine a compiled GP2040-CE firmware-only .bin and existing user and/or board storage area(s) "
+                    "or config .bin(s) into one file suitable for flashing onto a board.",
         parents=[core_parser],
     )
+    parser.add_argument('--replace-extra', action='store_true',
+                        help="if the firmware file is larger than the location of storage, perhaps because it's "
+                             "actually a full board dump, overwrite its config section with the config binary")
     parser.add_argument('firmware_filename', help=".bin file of a GP2040-CE firmware, probably from a build")
-    parser.add_argument('config_filename', help=".bin file of a GP2040-CE board's storage section or config w/footer")
-    parser.add_argument('new_binary_filename', help="output .bin file of the resulting firmware + storage")
+    board_config_group = parser.add_mutually_exclusive_group(required=False)
+    board_config_group.add_argument('--binary-board-config-filename',
+                                    help=".bin file of a GP2040-CE board config w/footer")
+    board_config_group.add_argument('--json-board-config-filename', help=".json file of a GP2040-CE board config")
+    user_config_group = parser.add_mutually_exclusive_group(required=False)
+    user_config_group.add_argument('--binary-user-config-filename',
+                                   help=".bin file of a GP2040-CE user config w/footer")
+    user_config_group.add_argument('--json-user-config-filename', help=".json file of a GP2040-CE user config")
+    output_group = parser.add_mutually_exclusive_group(required=True)
+    output_group.add_argument('--usb', action='store_true', help="write the resulting firmware + storage to USB")
+    output_group.add_argument('--new-filename', help="output .bin or .uf2 file of the resulting firmware + storage")
+    parser.add_argument('--backup', action='store_true', default=False,
+                        help="if the output file exists, move it to .old before writing")
 
     args, _ = parser.parse_known_args()
-    concatenate_firmware_and_storage_files(args.firmware_filename, args.config_filename, args.new_binary_filename)
+    concatenate_firmware_and_storage_files(args.firmware_filename,
+                                           binary_board_config_filename=args.binary_board_config_filename,
+                                           json_board_config_filename=args.json_board_config_filename,
+                                           binary_user_config_filename=args.binary_user_config_filename,
+                                           json_user_config_filename=args.json_user_config_filename,
+                                           combined_filename=args.new_filename, usb=args.usb,
+                                           replace_extra=args.replace_extra, backup=args.backup)
+
+
+def dump_gp2040ce():
+    """Copy the whole GP2040-CE section off of a BOOTSEL mode board."""
+    parser = argparse.ArgumentParser(
+        description="Read the GP2040-CE firmware + storage section off of a connected USB RP2040 in BOOTSEL mode.",
+        parents=[core_parser],
+    )
+    parser.add_argument('binary_filename', help="output .bin file of the resulting firmware + storage")
+
+    args, _ = parser.parse_known_args()
+    content, _, _ = get_gp2040ce_from_usb()
+    with open(args.binary_filename, 'wb') as out_file:
+        if args.binary_filename[-4:] == '.uf2':
+            # we must pad to storage start in order for the UF2 write addresses to make sense
+            out_file.write(storage.convert_binary_to_uf2([(0, content)]))
+        else:
+            out_file.write(content)
+
+
+def summarize_gp2040ce():
+    """Provide information on a dump or USB device."""
+    parser = argparse.ArgumentParser(
+        description="Read a file or USB device to determine what GP2040-CE parts are present.",
+        parents=[core_parser],
+    )
+    input_group = parser.add_mutually_exclusive_group(required=True)
+    input_group.add_argument('--usb', action='store_true', help="inspect the RP2040 device over USB")
+    input_group.add_argument('--filename', help="input .bin or .uf2 file to inspect")
+
+    args, _ = parser.parse_known_args()
+    if args.usb:
+        content, endpoint, _ = get_gp2040ce_from_usb()
+        print(f"USB device {hex(endpoint.device.idVendor)}:{hex(endpoint.device.idProduct)}:\n")
+    else:
+        content = storage.get_binary_from_file(args.filename)
+        print(f"File {args.filename}:\n")
+
+    gp2040ce_version = find_version_string_in_binary(content)
+    try:
+        board_config = storage.get_config(storage.get_board_storage_section(bytes(content)))
+        board_config_version = board_config.boardVersion if board_config.boardVersion else "NOT SPECIFIED"
+    except storage.ConfigReadError:
+        board_config_version = "NONE"
+    try:
+        user_config = storage.get_config(storage.get_user_storage_section(bytes(content)))
+        user_config_version = user_config.boardVersion if user_config.boardVersion else "NOT FOUND"
+    except storage.ConfigReadError:
+        user_config_version = "NONE"
+
+    print("GP2040-CE Information")
+    print(f"  detected GP2040-CE version:     {gp2040ce_version}")
+    print(f"  detected board config version:  {board_config_version}")
+    print(f"  detected user config version:   {user_config_version}")

gp2040ce_bintools/config_tree.css

@@ -0,0 +1,71 @@
+/**
+ * SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+ * SPDX-License-Identifier: GPL-3.0-or-later
+ */
+Tree {
+    padding: 1;
+}
+
+Button {
+    border: round gray;
+    content-align: center middle;
+    max-width: 50%;
+    height: 100%;
+    margin: 0 1;
+}
+
+EditScreen, MessageScreen, SaveAsScreen {
+    align: center middle;
+}
+
+EditScreen Label {
+    width: 100%;
+    height: 100%;
+    content-align: center middle;
+}
+
+#config_tree {
+    border: heavy gray;
+    padding: 1;
+    margin: 1;
+}
+
+.hidden {
+    visibility: hidden;
+}
+
+#edit-dialog, #save-as-dialog {
+    padding: 0 1;
+    grid-rows: 1fr 1fr 1fr 1fr;
+    width: 50%;
+    height: 50%;
+    border: tall gray 100%;
+}
+
+#message-dialog {
+    padding: 0 1;
+    grid-rows: 3fr 1fr;
+    max-width: 75%;
+    max-height: 75%;
+    border: tall gray 100%;
+}
+
+#button-container {
+    align: center middle;
+}
+
+#cancel-button-container {
+    align: left middle;
+}
+
+#confirm-button-container {
+    align: right middle;
+}
+
+#field-name, #field-input, #input-errors {
+    column-span: 2;
+}
+
+#field-input {
+    border: solid white;
+}

gp2040ce_bintools/gui.py

@@ -0,0 +1,474 @@
+"""GUI applications for working with binary files.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
+import argparse
+import logging
+from textwrap import dedent
+
+from google.protobuf import descriptor
+from google.protobuf.message import Message
+from rich.highlighter import ReprHighlighter
+from rich.text import Text
+from textual import on
+from textual.app import App, ComposeResult
+from textual.containers import Container, Grid, Horizontal
+from textual.logging import TextualHandler
+from textual.screen import ModalScreen
+from textual.validation import Length, Number
+from textual.widgets import Button, Footer, Header, Input, Label, Pretty, Select, TextArea, Tree
+from textual.widgets.tree import TreeNode
+
+from gp2040ce_bintools import _version, core_parser, handler
+from gp2040ce_bintools.builder import write_new_config_to_filename, write_new_config_to_usb
+from gp2040ce_bintools.rp2040 import get_bootsel_endpoints, read
+from gp2040ce_bintools.storage import (STORAGE_SIZE, USER_CONFIG_BOOTSEL_ADDRESS, ConfigReadError, get_config,
+                                       get_config_from_file, get_new_config)
+
+logger = logging.getLogger(__name__)
+
+
+class EditScreen(ModalScreen):
+    """Do an input prompt by way of an overlaid screen."""
+
+    def __init__(self, node: TreeNode, field_value: object, *args, **kwargs):
+        """Save the config field info for later usage."""
+        logger.debug("constructing EditScreen for %s", node.label)
+        self.node = node
+        parent_config, field_descriptor, _ = node.data
+        self.parent_config = parent_config
+        self.field_descriptor = field_descriptor
+        self.field_value = field_value
+        super().__init__(*args, **kwargs)
+
+    def compose(self) -> ComposeResult:
+        """Build the pop-up window with this result."""
+        if self.field_descriptor.type == descriptor.FieldDescriptor.TYPE_ENUM:
+            options = [(d.name, v) for v, d in self.field_descriptor.enum_type.values_by_number.items()]
+            self.input_field = Select(options, value=self.field_value, id='field-input')
+        elif self.field_descriptor.type in (descriptor.FieldDescriptor.TYPE_INT32,
+                                            descriptor.FieldDescriptor.TYPE_INT64,
+                                            descriptor.FieldDescriptor.TYPE_UINT32,
+                                            descriptor.FieldDescriptor.TYPE_UINT64):
+            self.input_field = Input(value=repr(self.field_value), validators=[Number()], id='field-input')
+        elif self.field_descriptor.type == descriptor.FieldDescriptor.TYPE_STRING:
+            self.input_field = Input(value=self.field_value, id='field-input')
+
+        yield Grid(
+            Container(Label(self.field_descriptor.full_name, id='field-name'), id='field-name-container'),
+            Container(self.input_field, id='input-field-container'),
+            Container(Pretty('', id='input-errors', classes='hidden'), id='error-container'),
+            Horizontal(Container(Button("Cancel", id='cancel-button'), id='cancel-button-container'),
+                       Container(Button("Confirm", id='confirm-button'), id='confirm-button-container'),
+                       id='button-container'),
+            id='edit-dialog',
+        )
+
+    @on(Input.Changed)
+    def show_invalid_reasons(self, event: Input.Changed) -> None:
+        """Update the UI to show why validation failed."""
+        if event.validation_result:
+            error_field = self.query_one(Pretty)
+            save_button = self.query_one('#confirm-button', Button)
+            if not event.validation_result.is_valid:
+                error_field.update(event.validation_result.failure_descriptions)
+                error_field.classes = ''
+                save_button.disabled = True
+            else:
+                error_field.update('')
+                error_field.classes = 'hidden'
+                save_button.disabled = False
+
+    def on_button_pressed(self, event: Button.Pressed) -> None:
+        """Process the button actions."""
+        if event.button.id == 'confirm-button':
+            logger.debug("calling _save")
+            self._save()
+        self.app.pop_screen()
+
+    def _save(self):
+        """Save the field value to the retained config item."""
+        if self.field_descriptor.type in (descriptor.FieldDescriptor.TYPE_INT32,
+                                          descriptor.FieldDescriptor.TYPE_INT64,
+                                          descriptor.FieldDescriptor.TYPE_UINT32,
+                                          descriptor.FieldDescriptor.TYPE_UINT64):
+            field_value = int(self.input_field.value)
+        else:
+            field_value = self.input_field.value
+        setattr(self.parent_config, self.field_descriptor.name, field_value)
+        logger.debug("parent config post-change: %s", self.parent_config)
+        self.node.set_label(pb_field_to_node_label(self.field_descriptor, field_value))
+
+
+class MessageScreen(ModalScreen):
+    """Simple screen for displaying messages."""
+
+    def __init__(self, text: str, *args, **kwargs):
+        """Store the message for later display."""
+        self.text = text
+        super().__init__(*args, **kwargs)
+
+    def compose(self) -> ComposeResult:
+        """Build the pop-up window with the desired message displayed."""
+        yield Grid(
+            Container(TextArea(self.text, id='message-text', read_only=True), id='text-container'),
+            Container(Button("OK", id='ok-button'), id='button-container'),
+            id='message-dialog',
+        )
+
+    def on_button_pressed(self, event: Button.Pressed) -> None:
+        """Process the button action (close the window)."""
+        self.app.pop_screen()
+
+
+class SaveAsScreen(ModalScreen):
+    """Present the option of saving the configuration as a new file."""
+
+    def __init__(self, config, *args, **kwargs):
+        """Initialize a filename argument to be populated."""
+        self.config = config
+        super().__init__(*args, **kwargs)
+
+    def compose(self) -> ComposeResult:
+        """Build the pop-up window prompting for the new filename to save the configuration as."""
+        self.filename_field = Input(value=None, id='field-input', validators=[Length(minimum=1)])
+        yield Grid(
+            Container(Label("Filename (.uf2, .bin, or .json) to write to:", id='field-name'),
+                      id='field-name-container'),
+            Container(self.filename_field, id='input-field-container'),
+            Container(Pretty('', id='input-errors', classes='hidden'), id='error-container'),
+            Horizontal(Container(Button("Cancel", id='cancel-button'), id='cancel-button-container'),
+                       Container(Button("Confirm", id='confirm-button'), id='confirm-button-container'),
+                       id='button-container'),
+            id='save-as-dialog',
+        )
+
+    @on(Input.Changed)
+    def show_invalid_reasons(self, event: Input.Changed) -> None:
+        """Update the UI to show why validation failed."""
+        if event.validation_result:
+            error_field = self.query_one(Pretty)
+            save_button = self.query_one('#confirm-button', Button)
+            if not event.validation_result.is_valid:
+                error_field.update(event.validation_result.failure_descriptions)
+                error_field.classes = ''
+                save_button.disabled = True
+            else:
+                error_field.update('')
+                error_field.classes = 'hidden'
+                save_button.disabled = False
+
+    def on_button_pressed(self, event: Button.Pressed) -> None:
+        """Process the button actions."""
+        if event.button.id == 'confirm-button':
+            logger.debug("calling _save")
+            self._save()
+        self.app.pop_screen()
+
+    def _save(self):
+        """Save the configuration to the specified file."""
+        write_new_config_to_filename(self.config, self.filename_field.value, inject=False)
+        self.notify(f"Saved to {self.filename_field.value}.", title="Configuration Saved")
+
+
+class ConfigEditor(App):
+    """Display the GP2040-CE configuration as a tree."""
+
+    BINDINGS = [
+        ('a', 'save_as', "Save As..."),
+        ('n', 'add_node', "Add Node"),
+        ('s', 'save', "Save Config"),
+        ('q', 'quit', "Quit"),
+        ('?', 'about', "About"),
+    ]
+    CSS_PATH = "config_tree.css"
+    TITLE = F"GP2040-CE Configuration Editor - {_version.version}"
+
+    def __init__(self, *args, **kwargs):
+        """Initialize config."""
+        # disable normal logging and enable console logging
+        logger.debug("reconfiguring logging...")
+        root = logging.getLogger()
+        root.setLevel(logging.DEBUG)
+        root.removeHandler(handler)
+        root.addHandler(TextualHandler())
+
+        self.config_filename = kwargs.pop('config_filename', None)
+        self.usb = kwargs.pop('usb', False)
+        self.whole_board = kwargs.pop('whole_board', False)
+        self.create_new = kwargs.pop('create_new', False)
+
+        super().__init__(*args, **kwargs)
+        self._load_config()
+
+        if self.usb:
+            self.source_name = (f"DEVICE ID {hex(self.endpoint_out.device.idVendor)}:"
+                                f"{hex(self.endpoint_out.device.idProduct)} "
+                                f"on bus {self.endpoint_out.device.bus} address {self.endpoint_out.device.address}")
+        else:
+            self.source_name = self.config_filename
+
+    def compose(self) -> ComposeResult:
+        """Compose the UI."""
+        yield Header()
+        yield Footer()
+        yield Tree("Root", id='config_tree')
+
+    def on_mount(self) -> None:
+        """Load the configuration object into the tree view."""
+        tree = self.query_one(Tree)
+
+        tree.root.data = (None, self.config.DESCRIPTOR, self.config)
+        tree.root.set_label(self.source_name)
+        missing_fields = [f for f in self.config.DESCRIPTOR.fields
+                          if f not in [fp for fp, vp in self.config.ListFields()]]
+        for field_descriptor, field_value in sorted(self.config.ListFields(), key=lambda f: f[0].name):
+            child_is_message = ConfigEditor._descriptor_is_message(field_descriptor)
+            ConfigEditor._add_node(tree.root, self.config, field_descriptor, field_value,
+                                   value_is_config=child_is_message)
+        for child_field_descriptor in sorted(missing_fields, key=lambda f: f.name):
+            child_is_message = ConfigEditor._descriptor_is_message(child_field_descriptor)
+            ConfigEditor._add_node(tree.root, self.config, child_field_descriptor,
+                                   getattr(self.config, child_field_descriptor.name),
+                                   value_is_config=child_is_message)
+        tree.root.expand()
+
+    def on_tree_node_selected(self, node_event: Tree.NodeSelected) -> None:
+        """Take the appropriate action for this type of node."""
+        self._modify_node(node_event.node)
+
+    def action_about(self) -> None:
+        """Display a help/about popup."""
+        self.push_screen(MessageScreen(dedent("""
+            gp2040ce-binary-tools - Tools for working with GP2040-CE firmware and storage binaries
+
+            Copyright © 2023 Brian S. Stephan <bss@incorporeal.org>
+            Made available WITHOUT ANY WARRANTY under the GNU General Public License, version 3 or later.
+        """)))
+
+    def action_add_node(self) -> None:
+        """Add a node to the tree item, if allowed by the tree and config section."""
+        tree = self.query_one(Tree)
+        current_node = tree.cursor_node
+
+        if not current_node or not current_node.allow_expand:
+            logger.debug("no node selected, or it does not allow expansion")
+            return
+
+        parent_config, field_descriptor, field_value = current_node.data
+        if not parent_config:
+            logger.debug("adding to the root is unsupported!")
+            return
+
+        if field_descriptor.type == descriptor.FieldDescriptor.TYPE_MESSAGE:
+            config = field_value
+        else:
+            config = getattr(parent_config, field_descriptor.name)
+        logger.debug("config: %s", config)
+        if hasattr(config, 'add'):
+            field_value = config.add()
+            actual_field_descriptor = parent_config.DESCRIPTOR.fields_by_name[field_descriptor.name]
+            logger.debug("adding new node %s", field_value.DESCRIPTOR.name)
+            ConfigEditor._add_node(current_node, config, actual_field_descriptor, field_value,
+                                   value_is_config=True)
+            current_node.expand()
+
+    def action_save(self) -> None:
+        """Save the configuration."""
+        if self.usb:
+            write_new_config_to_usb(self.config, self.endpoint_out, self.endpoint_in)
+            self.notify(f"Saved to {hex(self.endpoint_out.device.idVendor)}:"
+                        f"{hex(self.endpoint_out.device.idProduct)}.",
+                        title="Configuration Saved")
+        elif self.config_filename:
+            write_new_config_to_filename(self.config, self.config_filename, inject=self.whole_board)
+            self.notify(f"Saved to {self.config_filename}.",
+                        title="Configuration Saved")
+
+    def action_save_as(self) -> None:
+        """Present a new dialog to save the configuration as a new standalone file."""
+        self.push_screen(SaveAsScreen(self.config))
+
+    def action_quit(self) -> None:
+        """Quit the application."""
+        self.exit()
+
+    @staticmethod
+    def _add_node(parent_node: TreeNode, parent_config: Message,
+                  field_descriptor: descriptor.FieldDescriptor, field_value: object,
+                  value_is_config: bool = False, uninitialized: bool = False) -> None:
+        """Add a node to the overall tree, recursively.
+
+        Args:
+            parent_node: parent node to attach the new node(s) to
+            parent_config: the Config object parent. parent_config + field_descriptor.name = this node
+            field_descriptor: descriptor for the protobuf field
+            field_value: data to add to the parent node as new node(s)
+            value_is_config: get the config from the value rather than deriving it (important for repeated)
+            uninitialized: this node's data is from the spec and not the actual config, handle with care
+        """
+        # all nodes relate to their parent and retain info about themselves
+        this_node = parent_node.add("")
+        if uninitialized and 'google._upb._message.RepeatedCompositeContainer' in str(type(field_value)):
+            # python segfaults if I refer to/retain its actual, presumably uninitialized in C, value
+            logger.warning("PROBLEM: %s %s", type(field_value), field_value)
+            # WORKAROUND  BEGINS HERE
+            if not field_value:
+                x = field_value.add()
+                field_value.remove(x)
+            # WORKAROUND ENDS HERE
+        this_node.data = (parent_config, field_descriptor, field_value)
+
+        if uninitialized:
+            this_node.set_label(Text.from_markup("[red][b]NEW:[/b][/red] ") +
+                                pb_field_to_node_label(field_descriptor, field_value))
+        else:
+            this_node.set_label(pb_field_to_node_label(field_descriptor, field_value))
+
+        if ConfigEditor._descriptor_is_message(field_descriptor):
+            if value_is_config:
+                this_config = field_value
+            else:
+                this_config = getattr(parent_config, field_descriptor.name)
+
+            if hasattr(field_value, 'add'):
+                # support repeated
+                for child in field_value:
+                    child_is_message = ConfigEditor._descriptor_is_message(child.DESCRIPTOR)
+                    ConfigEditor._add_node(this_node, this_config, child.DESCRIPTOR, child,
+                                           value_is_config=child_is_message)
+            else:
+                # a message has stuff under it, recurse into it
+                missing_fields = [f for f in field_value.DESCRIPTOR.fields
+                                  if f not in [fp for fp, vp in field_value.ListFields()]]
+                for child_field_descriptor, child_field_value in sorted(field_value.ListFields(),
+                                                                        key=lambda f: f[0].name):
+                    child_is_message = ConfigEditor._descriptor_is_message(child_field_descriptor)
+                    ConfigEditor._add_node(this_node, this_config, child_field_descriptor, child_field_value,
+                                           value_is_config=child_is_message)
+                for child_field_descriptor in sorted(missing_fields, key=lambda f: f.name):
+                    child_is_message = ConfigEditor._descriptor_is_message(child_field_descriptor)
+                    ConfigEditor._add_node(this_node, this_config, child_field_descriptor,
+                                           getattr(this_config, child_field_descriptor.name), uninitialized=True,
+                                           value_is_config=child_is_message)
+        else:
+            # leaf node, stop here
+            this_node.allow_expand = False
+
+    @staticmethod
+    def _descriptor_is_message(desc: descriptor.Descriptor) -> bool:
+        return (getattr(desc, 'type', None) == descriptor.FieldDescriptor.TYPE_MESSAGE or
+                hasattr(desc, 'fields'))
+
+    def _modify_node(self, node: TreeNode) -> None:
+        """Modify the selected node by context of what type of config item it is."""
+        parent_config, field_descriptor, _ = node.data
+
+        # don't do anything special with selecting expandable nodes, since the framework already expands them
+        if (isinstance(field_descriptor, descriptor.Descriptor) or
+                field_descriptor.type == descriptor.FieldDescriptor.TYPE_MESSAGE):
+            return
+
+        field_value = getattr(parent_config, field_descriptor.name)
+        if field_descriptor.type == descriptor.FieldDescriptor.TYPE_BOOL:
+            # toggle bools inline
+            logger.debug("changing %s from %s...", field_descriptor.name, field_value)
+            field_value = not field_value
+            logger.debug("...to %s", field_value)
+            setattr(parent_config, field_descriptor.name, field_value)
+            node.data = (parent_config, field_descriptor, field_value)
+            node.set_label(pb_field_to_node_label(field_descriptor, field_value))
+            logger.debug(self.config)
+        else:
+            logger.debug("opening edit screen for %s", field_descriptor.name)
+            self.push_screen(EditScreen(node, field_value))
+
+    def _load_config(self):
+        """Based on how this was initialized, get the config in a variety of ways."""
+        if self.usb:
+            try:
+                self.endpoint_out, self.endpoint_in = get_bootsel_endpoints()
+                config_binary = read(self.endpoint_out, self.endpoint_in, USER_CONFIG_BOOTSEL_ADDRESS, STORAGE_SIZE)
+                self.config = get_config(bytes(config_binary))
+            except ConfigReadError:
+                if self.create_new:
+                    logger.warning("creating new config as the read one was invalid!")
+                    self.config = get_new_config()
+                else:
+                    raise
+        else:
+            try:
+                self.config = get_config_from_file(self.config_filename, whole_board=self.whole_board)
+            except FileNotFoundError:
+                if self.create_new:
+                    logger.warning("creating new config as the read one was invalid!")
+                    self.config = get_new_config()
+                else:
+                    raise
+            except ConfigReadError:
+                if self.create_new:
+                    logger.warning("creating new config as the read one was invalid!")
+                    self.config = get_new_config()
+                else:
+                    raise
+
+
+def pb_field_to_node_label(field_descriptor, field_value):
+    """Provide the pretty label for a tree node.
+
+    Args:
+        field_descriptor: protobuf field for determining the type
+        field_value: value to render
+    Returns:
+        prettified text representation of the field
+    """
+    highlighter = ReprHighlighter()
+    if hasattr(field_value, 'add'):
+        label = Text.from_markup(f"[b]{field_descriptor.name}[][/b]")
+    elif (getattr(field_descriptor, 'type', None) == descriptor.FieldDescriptor.TYPE_MESSAGE or
+          hasattr(field_descriptor, 'fields')):
+        label = Text.from_markup(f"[b]{field_descriptor.name}[/b]")
+    elif field_descriptor.type == descriptor.FieldDescriptor.TYPE_ENUM:
+        enum_selection = field_descriptor.enum_type.values_by_number[field_value].name
+        label = Text.assemble(
+            Text.from_markup(f"{field_descriptor.name} = "),
+            highlighter(enum_selection),
+        )
+    else:
+        label = Text.assemble(
+            Text.from_markup(f"{field_descriptor.name} = "),
+            highlighter(repr(field_value)),
+        )
+
+    return label
+
+
+############
+# COMMANDS #
+############
+
+
+def edit_config():
+    """Edit the configuration in an interactive fashion."""
+    parser = argparse.ArgumentParser(
+        description="Utilize a GUI to view and alter the contents of a GP2040-CE configuration.",
+        parents=[core_parser],
+    )
+    parser.add_argument('--whole-board', action='store_true', help="indicate the binary file is a whole board dump")
+    parser.add_argument('--new-if-not-found', action='store_true', default=True,
+                        help="if the file/USB device doesn't have a config section, start a new one (default: enabled)")
+    group = parser.add_mutually_exclusive_group(required=True)
+    group.add_argument('--usb', action='store_true', help="retrieve the config from a RP2040 board connected over USB "
+                                                          "and in BOOTSEL mode")
+    group.add_argument('--filename', help=".bin of a GP2040-CE's whole board dump if --whole-board is specified, or a"
+                                          ".bin file of a GP2040-CE board's config + footer or entire storage section; "
+                                          "if creating a new config, it can also be written in .uf2 format")
+    args, _ = parser.parse_known_args()
+
+    if args.usb:
+        app = ConfigEditor(usb=True, create_new=args.new_if_not_found)
+    else:
+        app = ConfigEditor(config_filename=args.filename, whole_board=args.whole_board,
+                           create_new=args.new_if_not_found)
+    app.run()

gp2040ce_bintools/proto_snapshot/config.proto

@@ -0,0 +1,849 @@
+syntax = "proto2";
+
+import "nanopb.proto";
+import "enums.proto";
+
+message GamepadOptions
+{
+    optional InputMode inputMode = 1;
+    optional DpadMode dpadMode = 2;
+    optional SOCDMode socdMode = 3;
+    optional bool invertXAxis = 4;
+    optional bool invertYAxis = 5;
+    optional bool switchTpShareForDs4 = 6;
+    optional bool lockHotkeys = 7;
+    optional bool fourWayMode = 8;
+    optional uint32 profileNumber = 9;
+    optional PS4ControllerType ps4ControllerType = 10;
+    optional uint32 debounceDelay = 11;
+    optional int32 inputModeB1 = 12;
+    optional int32 inputModeB2 = 13;
+    optional int32 inputModeB3 = 14;
+    optional int32 inputModeB4 = 15;
+    optional int32 inputModeL1 = 16;
+    optional int32 inputModeL2 = 17;
+    optional int32 inputModeR1 = 18;
+    optional int32 inputModeR2 = 19;
+    optional bool ps4ReportHack = 20 [deprecated = true];
+    optional InputModeAuthType ps4AuthType = 21;
+    optional InputModeAuthType ps5AuthType = 22;
+    optional InputModeAuthType xinputAuthType = 23;
+    optional PS4ControllerIDMode ps4ControllerIDMode = 24;
+}
+
+message KeyboardMapping
+{
+    optional uint32 keyDpadUp = 1;
+    optional uint32 keyDpadDown = 2;
+    optional uint32 keyDpadLeft = 3;
+    optional uint32 keyDpadRight = 4;
+    optional uint32 keyButtonB1 = 5;
+    optional uint32 keyButtonB2 = 6;
+    optional uint32 keyButtonB3 = 7;
+    optional uint32 keyButtonB4 = 8;
+    optional uint32 keyButtonL1 = 9;
+    optional uint32 keyButtonR1 = 10;
+    optional uint32 keyButtonL2 = 11;
+    optional uint32 keyButtonR2 = 12;
+    optional uint32 keyButtonS1 = 13;
+    optional uint32 keyButtonS2 = 14;
+    optional uint32 keyButtonL3 = 15;
+    optional uint32 keyButtonR3 = 16;
+    optional uint32 keyButtonA1 = 17;
+    optional uint32 keyButtonA2 = 18;
+    optional uint32 keyButtonA3 = 19;
+    optional uint32 keyButtonA4 = 20;
+    optional uint32 keyButtonE1 = 21;
+    optional uint32 keyButtonE2 = 22;
+    optional uint32 keyButtonE3 = 23;
+    optional uint32 keyButtonE4 = 24;
+    optional uint32 keyButtonE5 = 25;
+    optional uint32 keyButtonE6 = 26;
+    optional uint32 keyButtonE7 = 27;
+    optional uint32 keyButtonE8 = 28;
+    optional uint32 keyButtonE9 = 29;
+    optional uint32 keyButtonE10 = 30;
+    optional uint32 keyButtonE11 = 31;
+    optional uint32 keyButtonE12 = 32;
+
+}
+
+message HotkeyEntry
+{
+    optional uint32 dpadMask = 1;
+    optional GamepadHotkey action = 2;
+    optional uint32 buttonsMask = 3;
+    optional uint32 auxMask = 4;
+}
+
+message HotkeyOptions
+{
+    optional HotkeyEntry hotkey01 = 1;
+    optional HotkeyEntry hotkey02 = 2;
+    optional HotkeyEntry hotkey03 = 3;
+    optional HotkeyEntry hotkey04 = 4;
+    optional HotkeyEntry hotkey05 = 5;
+    optional HotkeyEntry hotkey06 = 6;
+    optional HotkeyEntry hotkey07 = 7;
+    optional HotkeyEntry hotkey08 = 8;
+    optional HotkeyEntry hotkey09 = 9;
+    optional HotkeyEntry hotkey10 = 10;
+    optional HotkeyEntry hotkey11 = 11;
+    optional HotkeyEntry hotkey12 = 12;
+    optional HotkeyEntry hotkey13 = 13;
+    optional HotkeyEntry hotkey14 = 14;
+    optional HotkeyEntry hotkey15 = 15;
+    optional HotkeyEntry hotkey16 = 16;
+}
+
+message PeripheralOptions
+{
+    message I2COptions {
+        optional bool enabled = 1;
+        optional int32 sda = 2;
+        optional int32 scl = 3;
+        optional uint32 speed = 4;
+    }
+
+    message SPIOptions {
+        optional bool enabled = 1;
+        optional int32 rx = 2;
+        optional int32 cs = 3;
+        optional int32 sck = 4;
+        optional int32 tx = 5;
+    }
+
+    message USBOptions {
+        optional bool enabled = 1;
+        optional int32 dp = 2;
+        optional int32 enable5v = 3;
+        optional uint32 order = 4;
+    }
+
+    optional I2COptions blockI2C0 = 1;
+    optional I2COptions blockI2C1 = 2;
+    optional SPIOptions blockSPI0 = 3;
+    optional SPIOptions blockSPI1 = 4;
+    optional USBOptions blockUSB0 = 5;
+}
+
+message ForcedSetupOptions
+{
+    optional ForcedSetupMode mode = 1;
+};
+
+message ButtonLayoutParamsCommon
+{
+    optional int32 startX = 1;
+    optional int32 startY = 2;
+    optional int32 buttonRadius = 3;
+    optional int32 buttonPadding = 4;
+}
+
+message ButtonLayoutParamsLeft
+{
+    optional ButtonLayout layout = 1;
+    optional ButtonLayoutParamsCommon common = 2;
+}
+
+message ButtonLayoutParamsRight
+{
+    optional ButtonLayoutRight layout = 1;
+    optional ButtonLayoutParamsCommon common = 2;
+}
+
+message ButtonLayoutCustomOptions
+{
+    optional ButtonLayoutParamsLeft paramsLeft = 1;
+    optional ButtonLayoutParamsRight paramsRight = 2;
+}
+
+message PinMappings
+{
+    optional int32 pinDpadUp = 1;
+    optional int32 pinDpadDown = 2;
+    optional int32 pinDpadLeft = 3;
+    optional int32 pinDpadRight = 4;
+    optional int32 pinButtonB1 = 5;
+    optional int32 pinButtonB2 = 6;
+    optional int32 pinButtonB3 = 7;
+    optional int32 pinButtonB4 = 8;
+    optional int32 pinButtonL1 = 9;
+    optional int32 pinButtonR1 = 10;
+    optional int32 pinButtonL2 = 11;
+    optional int32 pinButtonR2 = 12;
+    optional int32 pinButtonS1 = 13;
+    optional int32 pinButtonS2 = 14;
+    optional int32 pinButtonL3 = 15;
+    optional int32 pinButtonR3 = 16;
+    optional int32 pinButtonA1 = 17;
+    optional int32 pinButtonA2 = 18;
+    optional int32 pinButtonFn = 19;
+}
+
+message GpioMappingInfo
+{
+    optional GpioAction action = 1;
+    optional GpioDirection direction = 2;
+    optional uint32 customDpadMask = 3;
+    optional uint32 customButtonMask = 4;
+}
+
+message GpioMappings
+{
+    repeated GpioMappingInfo pins = 1 [(nanopb).max_count = 30];
+    optional string profileLabel = 2 [(nanopb).max_length = 16];
+    optional bool enabled = 3 [default = false];
+}
+
+
+message AlternativePinMappings
+{
+    optional int32 pinButtonB1 = 1;
+    optional int32 pinButtonB2 = 2;
+    optional int32 pinButtonB3 = 3;
+    optional int32 pinButtonB4 = 4;
+    optional int32 pinButtonL1 = 5;
+    optional int32 pinButtonR1 = 6;
+    optional int32 pinButtonL2 = 7;
+    optional int32 pinButtonR2 = 8;
+    optional int32 pinDpadUp = 9;
+    optional int32 pinDpadDown = 10;
+    optional int32 pinDpadLeft = 11;
+    optional int32 pinDpadRight = 12;
+}
+
+
+message ProfileOptions
+{
+    repeated AlternativePinMappings deprecatedAlternativePinMappings = 1 [(nanopb).max_count = 3, deprecated = true];
+    repeated GpioMappings gpioMappingsSets = 2 [(nanopb).max_count = 3];
+}
+
+message DisplayOptions
+{
+    optional bool enabled = 1;
+    
+    optional int32 deprecatedI2cBlock = 2 [deprecated = true];
+    optional int32 deprecatedI2cSDAPin = 3 [deprecated = true];
+    optional int32 deprecatedI2cSCLPin = 4 [deprecated = true];
+    optional int32 deprecatedI2cAddress = 5 [deprecated = true];
+    optional int32 deprecatedI2cSpeed = 6 [deprecated = true];
+
+    optional ButtonLayout buttonLayout = 7;
+    optional ButtonLayoutRight buttonLayoutRight = 8;
+    optional ButtonLayoutCustomOptions buttonLayoutCustomOptions = 9;
+
+    optional SplashMode splashMode = 10;
+    optional SplashChoice splashChoice = 11;
+    optional int32 splashDuration = 12;
+    optional bytes splashImage = 13 [(nanopb).max_size = 1024];
+
+    optional int32 size = 14;
+    optional int32 flip = 15;
+    optional bool invert = 16;
+
+    optional int32 displaySaverTimeout = 17;
+    optional bool turnOffWhenSuspended = 18;
+}
+
+message LEDOptions
+{
+    optional int32 dataPin = 1;
+    optional LEDFormat_Proto ledFormat = 2;
+    optional ButtonLayout ledLayout = 3;
+    optional uint32 ledsPerButton = 4;
+    optional uint32 brightnessMaximum = 5;
+    optional uint32 brightnessSteps = 6;
+
+    optional int32 indexUp = 7;
+    optional int32 indexDown = 8;
+    optional int32 indexLeft = 9;
+    optional int32 indexRight = 10;
+    optional int32 indexB1 = 11;
+    optional int32 indexB2 = 12;
+    optional int32 indexB3 = 13;
+    optional int32 indexB4 = 14;
+    optional int32 indexL1 = 15;
+    optional int32 indexR1 = 16;
+    optional int32 indexL2 = 17;
+    optional int32 indexR2 = 18;
+    optional int32 indexS1 = 19;
+    optional int32 indexS2 = 20;
+    optional int32 indexL3 = 21;
+    optional int32 indexR3 = 22;
+    optional int32 indexA1 = 23;
+    optional int32 indexA2 = 24;
+
+    optional PLEDType pledType = 25;
+    optional int32 pledPin1 = 26;
+    optional int32 pledPin2 = 27;
+    optional int32 pledPin3 = 28;
+    optional int32 pledPin4 = 29;
+    optional uint32 pledColor = 30;
+
+    optional bool turnOffWhenSuspended = 31;
+
+    optional int32 pledIndex1 = 32;
+    optional int32 pledIndex2 = 33;
+    optional int32 pledIndex3 = 34;
+    optional int32 pledIndex4 = 35;
+};
+
+// This has to be kept in sync with AnimationOptions in AnimationStation.hpp
+message AnimationOptions_Proto
+{
+    optional uint32 baseAnimationIndex = 1;
+    optional uint32 brightness = 2;
+    optional uint32 staticColorIndex = 3;
+    optional uint32 buttonColorIndex = 4;
+    optional int32 chaseCycleTime = 5;
+    optional int32 rainbowCycleTime = 6;
+    optional uint32 themeIndex = 7;
+
+    optional bool hasCustomTheme = 8;
+    optional uint32 customThemeUp = 9;
+    optional uint32 customThemeDown = 10;
+    optional uint32 customThemeLeft = 11;
+    optional uint32 customThemeRight = 12;
+    optional uint32 customThemeB1 = 13;
+    optional uint32 customThemeB2 = 14;
+    optional uint32 customThemeB3 = 15;
+    optional uint32 customThemeB4 = 16;
+    optional uint32 customThemeL1 = 17;
+    optional uint32 customThemeR1 = 18;
+    optional uint32 customThemeL2 = 19;
+    optional uint32 customThemeR2 = 20;
+    optional uint32 customThemeS1 = 21;
+    optional uint32 customThemeS2 = 22;
+    optional uint32 customThemeL3 = 23;
+    optional uint32 customThemeR3 = 24;
+    optional uint32 customThemeA1 = 25;
+    optional uint32 customThemeA2 = 26;
+    optional uint32 customThemeUpPressed = 27;
+    optional uint32 customThemeDownPressed = 28;
+    optional uint32 customThemeLeftPressed = 29;
+    optional uint32 customThemeRightPressed = 30;
+    optional uint32 customThemeB1Pressed = 31;
+    optional uint32 customThemeB2Pressed = 32;
+    optional uint32 customThemeB3Pressed = 33;
+    optional uint32 customThemeB4Pressed = 34;
+    optional uint32 customThemeL1Pressed = 35;
+    optional uint32 customThemeR1Pressed = 36;
+    optional uint32 customThemeL2Pressed = 37;
+    optional uint32 customThemeR2Pressed = 38;
+    optional uint32 customThemeS1Pressed = 39;
+    optional uint32 customThemeS2Pressed = 40;
+    optional uint32 customThemeL3Pressed = 41;
+    optional uint32 customThemeR3Pressed = 42;
+    optional uint32 customThemeA1Pressed = 43;
+    optional uint32 customThemeA2Pressed = 44;
+    optional uint32 buttonPressColorCooldownTimeInMs = 45;
+}
+
+message BootselButtonOptions
+{
+    optional bool enabled = 1;
+    optional uint32 buttonMap = 2;
+}
+
+message OnBoardLedOptions
+{
+    optional OnBoardLedMode mode = 1;
+    optional bool enabled = 2;
+}
+
+message AnalogOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 analogAdc1PinX = 2;
+    optional int32 analogAdc1PinY = 3;
+    optional bool forced_circularity = 4;
+    optional uint32 inner_deadzone = 5;
+    optional int32 analogAdc2PinX = 6;
+    optional int32 analogAdc2PinY = 7;
+    optional DpadMode analogAdc1Mode = 8;
+    optional DpadMode analogAdc2Mode = 9;
+    optional InvertMode analogAdc1Invert = 10;
+    optional InvertMode analogAdc2Invert = 11;
+    optional bool auto_calibrate = 12;
+    optional uint32 outer_deadzone = 13;
+    optional bool analog_smoothing = 14;
+    optional float smoothing_factor = 15;
+    optional uint32 analog_error = 16;
+}
+
+message TurboOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 deprecatedButtonPin = 2 [deprecated = true];
+    optional int32 ledPin = 3;
+    optional uint32 shotCount = 4;
+    optional int32 shmupDialPin = 5;
+
+    optional bool shmupModeEnabled = 6;
+    optional uint32 shmupAlwaysOn1 = 7;
+    optional uint32 shmupAlwaysOn2 = 8;
+    optional uint32 shmupAlwaysOn3 = 9;
+    optional uint32 shmupAlwaysOn4 = 10;
+    optional int32 shmupBtn1Pin = 11;
+    optional int32 shmupBtn2Pin = 12;
+    optional int32 shmupBtn3Pin = 13;
+    optional int32 shmupBtn4Pin = 14;
+    optional uint32 shmupBtnMask1 = 15;
+    optional uint32 shmupBtnMask2 = 16;
+    optional uint32 shmupBtnMask3 = 17;
+    optional uint32 shmupBtnMask4 = 18;
+    optional ShmupMixMode shmupMixMode = 19;
+}
+
+message SliderOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 deprecatedPinSliderOne = 2 [deprecated = true];
+    optional int32 deprecatedPinSliderTwo = 3 [deprecated = true];
+    optional DpadMode deprecatedModeOne = 4 [deprecated = true];
+    optional DpadMode deprecatedModeTwo = 5 [deprecated = true];
+    optional DpadMode deprecatedModeDefault = 6;
+}
+
+message SOCDSliderOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 deprecatedPinOne = 2 [deprecated = true];
+    optional int32 deprecatedPinTwo = 3 [deprecated = true];
+
+    optional SOCDMode modeDefault = 4;
+    optional SOCDMode deprecatedModeOne = 5 [deprecated = true];
+    optional SOCDMode deprecatedModeTwo = 6 [deprecated = true];
+}
+
+message ReverseOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 buttonPin = 2;
+    optional int32 ledPin = 3;
+
+    optional uint32 actionUp = 4;
+    optional uint32 actionDown = 5;
+    optional uint32 actionLeft = 6;
+    optional uint32 actionRight = 7;
+}
+
+message AnalogADS1219Options
+{
+    optional bool enabled = 1;
+    
+    optional int32 deprecatedI2cBlock = 2 [deprecated = true];
+    optional int32 deprecatedI2cSDAPin = 3 [deprecated = true];
+    optional int32 deprecatedI2cSCLPin = 4 [deprecated = true];
+    optional int32 deprecatedI2cAddress = 5 [deprecated = true];
+    optional int32 deprecatedI2cSpeed = 6 [deprecated = true];
+}
+
+message AnalogADS1256Options
+{
+    optional bool enabled = 1;
+
+    optional int32 spiBlock = 2;
+    optional int32 csPin = 3;
+    optional int32 drdyPin = 4;
+    optional float avdd = 5;
+    optional bool enableTriggers = 6;
+}
+
+message DualDirectionalOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 deprecatedUpPin = 2 [deprecated = true];
+    optional int32 deprecatedDownPin = 3 [deprecated = true];
+    optional int32 deprecatedLeftPin = 4 [deprecated = true];
+    optional int32 deprecatedRightPin = 5 [deprecated = true];
+
+    optional DpadMode dpadMode = 6;
+    optional DualDirectionalCombinationMode combineMode = 7;
+    optional bool fourWayMode = 8;
+}
+
+message TiltOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 tilt1Pin = 2;
+    optional int32 tilt2Pin = 3;
+    optional int32 deprecatedTiltFunctionPin = 4 [deprecated = true];
+    optional int32 tiltLeftAnalogUpPin = 5;
+    optional int32 tiltLeftAnalogDownPin = 6;
+    optional int32 tiltLeftAnalogLeftPin = 7;
+    optional int32 tiltLeftAnalogRightPin = 8;
+    optional int32 tiltRightAnalogUpPin = 9;
+    optional int32 tiltRightAnalogDownPin = 10;
+    optional int32 tiltRightAnalogLeftPin = 11;
+    optional int32 tiltRightAnalogRightPin = 12;
+
+    optional SOCDMode tiltSOCDMode = 13;
+
+    optional int32 factorTilt1LeftX =  14;
+    optional int32 factorTilt1LeftY =  15;
+    optional int32 factorTilt1RightX =  16;
+    optional int32 factorTilt1RightY =  17;
+    optional int32 factorTilt2LeftX =  18;
+    optional int32 factorTilt2LeftY =  19;
+    optional int32 factorTilt2RightX =  20;
+    optional int32 factorTilt2RightY =  21;
+}
+
+message BuzzerOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 pin = 2;
+    optional uint32 volume = 3;
+    optional int32 enablePin = 4;
+}
+
+message ExtraButtonOptions
+{
+    optional bool enabled = 1;
+
+    optional int32 pin = 2;
+    optional uint32 buttonMap = 3;
+}
+
+message PlayerNumberOptions
+{
+    optional bool enabled = 1;
+    optional uint32 number = 2;
+}
+
+message PS4Options
+{
+    optional bool enabled = 1;
+    optional bytes serial = 2 [(nanopb).max_size = 16];
+    optional bytes signature = 3 [(nanopb).max_size = 256];
+    optional bytes rsaN = 4 [(nanopb).max_size = 256];
+    optional bytes rsaE = 5 [(nanopb).max_size = 4];
+    optional bytes rsaD = 6 [(nanopb).max_size = 256];
+    optional bytes rsaP = 7 [(nanopb).max_size = 128];
+    optional bytes rsaQ = 8 [(nanopb).max_size = 128];
+    optional bytes rsaDP = 9 [(nanopb).max_size = 128];
+    optional bytes rsaDQ = 10 [(nanopb).max_size = 128];
+    optional bytes rsaQP = 11 [(nanopb).max_size = 128];
+    optional bytes rsaRN = 12 [(nanopb).max_size = 256];
+}
+
+message PSPassthroughOptions
+{
+    optional bool enabled = 1  [deprecated = true];
+    optional int32 deprecatedPinDplus = 2 [deprecated = true];
+    optional int32 deprecatedPin5V = 3 [deprecated = true];
+}
+
+message XBOnePassthroughOptions
+{
+    optional bool enabled = 1  [deprecated = true];
+}
+
+message WiiOptions
+{
+    message AnalogAxis
+    {
+        optional int32 axisType = 1;
+        optional int32 minRange = 2;
+        optional int32 maxRange = 3;
+    }
+
+    message StickOptions
+    {
+        optional AnalogAxis x = 1;
+        optional AnalogAxis y = 2;
+    }
+
+    message NunchukOptions
+    {
+        optional int32 buttonC = 1;
+        optional int32 buttonZ = 2;
+        optional StickOptions stick = 3;
+    }
+
+    message ClassicOptions
+    {
+        optional int32 buttonA = 1;
+        optional int32 buttonB = 2;
+        optional int32 buttonX = 3;
+        optional int32 buttonY = 4;
+        optional int32 buttonL = 5;
+        optional int32 buttonZL = 6;
+        optional int32 buttonR = 7;
+        optional int32 buttonZR = 8;
+        optional int32 buttonMinus = 9;
+        optional int32 buttonPlus = 10;
+        optional int32 buttonHome = 11;
+        optional int32 buttonUp = 12;
+        optional int32 buttonDown = 13;
+        optional int32 buttonLeft = 14;
+        optional int32 buttonRight = 15;
+        optional StickOptions rightStick = 17;
+        optional StickOptions leftStick = 16;
+        optional AnalogAxis leftTrigger = 18;
+        optional AnalogAxis rightTrigger = 19;
+    }
+
+    message TaikoOptions
+    {
+        optional int32 buttonKatLeft = 1;
+        optional int32 buttonKatRight = 2;
+        optional int32 buttonDonLeft = 3;
+        optional int32 buttonDonRight = 4;
+    }
+
+    message GuitarOptions
+    {
+        optional int32 buttonRed = 1;
+        optional int32 buttonGreen = 2;
+        optional int32 buttonYellow = 3;
+        optional int32 buttonBlue = 4;
+        optional int32 buttonOrange = 5;
+        optional int32 buttonPedal = 6;
+        optional int32 buttonMinus = 7;
+        optional int32 buttonPlus = 8;
+        optional int32 strumUp = 9;
+        optional int32 strumDown = 10;
+        optional StickOptions stick = 11;
+        optional AnalogAxis whammyBar = 12;
+    }
+
+    message DrumOptions
+    {
+        optional int32 buttonRed = 1;
+        optional int32 buttonGreen = 2;
+        optional int32 buttonYellow = 3;
+        optional int32 buttonBlue = 4;
+        optional int32 buttonOrange = 5;
+        optional int32 buttonPedal = 6;
+        optional int32 buttonMinus = 7;
+        optional int32 buttonPlus = 8;
+        optional StickOptions stick = 9;
+    }
+
+    message TurntableOptions
+    {
+        optional int32 buttonLeftRed = 1;
+        optional int32 buttonLeftGreen = 2;
+        optional int32 buttonLeftBlue = 3;
+        optional int32 buttonRightRed = 4;
+        optional int32 buttonRightGreen = 5;
+        optional int32 buttonRightBlue = 6;
+        optional int32 buttonMinus = 7;
+        optional int32 buttonPlus = 8;
+        optional int32 buttonEuphoria = 9;
+        optional StickOptions stick = 10;
+        optional AnalogAxis leftTurntable = 11;
+        optional AnalogAxis rightTurntable = 12;
+        optional AnalogAxis effects = 13;
+        optional AnalogAxis fader = 14;
+    }
+
+    message ControllerOptions
+    {
+        optional NunchukOptions nunchuk = 1;
+        optional ClassicOptions classic = 2;
+        optional TaikoOptions taiko = 3;
+        optional GuitarOptions guitar = 4;
+        optional DrumOptions drum = 5;
+        optional TurntableOptions turntable = 6;
+    }
+
+    optional bool enabled = 1;
+    optional int32 deprecatedI2cBlock = 2 [deprecated = true];
+    optional int32 deprecatedI2cSDAPin = 3 [deprecated = true];
+    optional int32 deprecatedI2cSCLPin = 4 [deprecated = true];
+    optional int32 deprecatedI2cSpeed = 5 [deprecated = true];
+
+    optional ControllerOptions controllers = 6;
+}
+
+message SNESOptions
+{
+    optional bool enabled = 1;
+    optional int32 clockPin = 2;
+    optional int32 latchPin = 3;
+    optional int32 dataPin = 4;
+}
+
+message KeyboardHostOptions
+{
+    optional bool enabled = 1;
+    optional int32 deprecatedPinDplus = 2 [deprecated = true];
+    optional KeyboardMapping mapping = 3;
+    optional int32 deprecatedPin5V = 4 [deprecated = true];
+    optional uint32 mouseLeft = 5;
+    optional uint32 mouseMiddle = 6;
+    optional uint32 mouseRight = 7;
+}
+
+message FocusModeOptions
+{
+    optional bool enabled = 1;
+    optional int32 pin = 2;
+    optional int32 buttonLockMask = 3;
+    optional bool oledLockEnabled = 4;
+    optional bool rgbLockEnabled = 5;
+    optional bool buttonLockEnabled = 6;
+    optional bool macroLockEnabled = 7;
+}
+
+message MacroInput
+{
+    optional uint32 buttonMask = 1;
+    optional uint32 duration = 2;
+    optional uint32 waitDuration = 3 [default = 0];
+}
+
+message Macro
+{
+    optional MacroType macroType = 1;
+    optional string macroLabel = 2 [(nanopb).max_length = 64];
+    repeated MacroInput macroInputs = 3 [(nanopb).max_count = 30];
+    optional bool enabled = 4;
+    optional bool useMacroTriggerButton = 5;
+    optional int32 deprecatedMacroTriggerPin = 6 [deprecated = true];
+    optional uint32 macroTriggerButton = 7;
+    optional bool exclusive = 8 [default = true];
+    optional bool interruptible = 9 [default = true];
+    optional bool showFrames = 10 [default = false];
+}
+
+message MacroOptions
+{
+    optional bool enabled = 1;
+    optional int32 deprecatedPin = 2 [deprecated = true];
+    repeated Macro macroList = 3 [(nanopb).max_count = 6];
+    optional bool macroBoardLedEnabled = 4;
+}
+
+message InputHistoryOptions
+{
+    optional bool enabled = 1;
+    optional uint32 length = 2;
+    optional uint32 col = 3;
+    optional uint32 row = 4;
+}
+
+message RotaryPinOptions
+{
+    optional bool enabled = 1;
+    optional int32 pinA = 2;
+    optional int32 pinB = 3;
+    optional RotaryEncoderPinMode mode = 4;
+    optional uint32 pulsesPerRevolution = 5;
+    optional uint32 resetAfter = 6;
+    optional bool allowWrapAround = 7;
+    optional float multiplier = 8;
+}
+
+message RotaryOptions
+{
+    optional bool enabled = 1;
+    optional RotaryPinOptions encoderOne = 2;
+    optional RotaryPinOptions encoderTwo = 3;
+}
+
+message PCF8575Options
+{
+    optional bool enabled = 1;
+    optional int32 deprecatedI2cBlock = 2 [deprecated = true];
+    repeated GpioMappingInfo pins = 3 [(nanopb).max_count = 16];
+}
+
+message DRV8833RumbleOptions
+{
+		optional bool enabled = 1;
+
+		optional int32 leftMotorPin = 2;
+		optional int32 rightMotorPin = 3;
+		optional int32 motorSleepPin = 4;
+		optional uint32 pwmFrequency = 5;
+		optional float dutyMin = 6;
+		optional float dutyMax = 7;
+}
+
+message ReactiveLEDInfo
+{
+    optional int32 pin = 1;
+    optional GpioAction action = 2;
+    optional ReactiveLEDMode modeDown = 3;
+    optional ReactiveLEDMode modeUp = 4;
+}
+
+message ReactiveLEDOptions
+{
+    optional bool enabled = 1;
+    repeated ReactiveLEDInfo leds = 2 [(nanopb).max_count = 10];
+}
+
+message AddonOptions
+{
+    optional BootselButtonOptions bootselButtonOptions = 1;
+    optional OnBoardLedOptions onBoardLedOptions = 2;
+    optional AnalogOptions analogOptions = 3;
+    optional TurboOptions turboOptions = 4;
+    optional SliderOptions deprecatedSliderOptions = 5;
+    optional ReverseOptions reverseOptions = 6;
+    optional AnalogADS1219Options analogADS1219Options = 7;
+    optional DualDirectionalOptions dualDirectionalOptions = 8;
+    optional BuzzerOptions buzzerOptions = 9;
+    optional ExtraButtonOptions deprecatedExtraButtonOptions = 10 [deprecated = true];
+    optional PlayerNumberOptions playerNumberOptions = 11;
+    optional PS4Options ps4Options = 12 [(nanopb).disallow_export = true];
+    optional WiiOptions wiiOptions = 13;
+    optional SOCDSliderOptions socdSliderOptions = 14;
+    optional SNESOptions snesOptions = 15;
+    optional FocusModeOptions focusModeOptions = 16;
+    optional KeyboardHostOptions keyboardHostOptions = 17;
+    optional TiltOptions tiltOptions = 18;
+    optional PSPassthroughOptions psPassthroughOptions = 19 [deprecated = true];
+    optional MacroOptions macroOptions = 20;
+    optional InputHistoryOptions inputHistoryOptions = 21;
+    optional XBOnePassthroughOptions xbonePassthroughOptions = 22 [deprecated = true];
+    optional AnalogADS1256Options analogADS1256Options = 23;
+    optional RotaryOptions rotaryOptions = 24;
+    optional PCF8575Options pcf8575Options = 25;
+    optional DRV8833RumbleOptions drv8833RumbleOptions = 26;
+    optional ReactiveLEDOptions reactiveLEDOptions = 27;
+}
+
+message MigrationHistory
+{
+    optional bool hotkeysMigrated = 1 [default = false];
+    optional bool gpioMappingsMigrated = 2 [default = false];
+    optional bool buttonProfilesMigrated = 3 [default = false];
+    optional bool profileEnabledFlagsMigrated = 4 [default = false];
+}
+
+message Config
+{
+    optional string boardVersion = 1 [(nanopb).max_length = 31];
+
+    optional GamepadOptions gamepadOptions = 2;
+    optional HotkeyOptions hotkeyOptions = 3;
+    optional PinMappings deprecatedPinMappings = 4 [deprecated = true];
+    optional KeyboardMapping keyboardMapping = 5;
+    optional DisplayOptions displayOptions = 6;
+    optional LEDOptions ledOptions = 7;
+    optional AnimationOptions_Proto animationOptions = 8;
+    optional AddonOptions addonOptions = 9;
+    optional ForcedSetupOptions forcedSetupOptions = 10;
+    optional ProfileOptions profileOptions = 11;
+
+    optional string boardConfig = 12 [(nanopb).max_length = 63];
+    optional GpioMappings gpioMappings = 13;
+    optional MigrationHistory migrations = 14;
+    optional PeripheralOptions peripheralOptions = 15;
+}

gp2040ce_bintools/proto_snapshot/enums.proto

@@ -0,0 +1,425 @@
+syntax = "proto2";
+
+import "nanopb.proto";
+
+enum ButtonLayout
+{
+    option (nanopb_enumopt).long_names = false;
+
+    BUTTON_LAYOUT_STICK = 0;
+    BUTTON_LAYOUT_STICKLESS = 1;
+    BUTTON_LAYOUT_BUTTONS_ANGLED = 2;
+    BUTTON_LAYOUT_BUTTONS_BASIC = 3;
+    BUTTON_LAYOUT_KEYBOARD_ANGLED = 4;
+    BUTTON_LAYOUT_KEYBOARDA = 5;
+    BUTTON_LAYOUT_DANCEPADA = 6;
+    BUTTON_LAYOUT_TWINSTICKA = 7;
+    BUTTON_LAYOUT_BLANKA = 8;
+    BUTTON_LAYOUT_VLXA = 9;
+    BUTTON_LAYOUT_FIGHTBOARD_STICK = 10;
+    BUTTON_LAYOUT_FIGHTBOARD_MIRRORED = 11;
+    BUTTON_LAYOUT_CUSTOMA = 12;
+    BUTTON_LAYOUT_OPENCORE0WASDA = 13;
+    BUTTON_LAYOUT_STICKLESS_13 = 14;
+    BUTTON_LAYOUT_STICKLESS_16 = 15;
+    BUTTON_LAYOUT_STICKLESS_14 = 16;
+    BUTTON_LAYOUT_DANCEPAD_DDR_LEFT = 17;
+    BUTTON_LAYOUT_DANCEPAD_DDR_SOLO = 18;
+    BUTTON_LAYOUT_DANCEPAD_PIU_LEFT = 19;
+    BUTTON_LAYOUT_POPN_A = 20;
+    BUTTON_LAYOUT_TAIKO_A = 21;
+    BUTTON_LAYOUT_BM_TURNTABLE_A = 22;
+    BUTTON_LAYOUT_BM_5KEY_A = 23;
+    BUTTON_LAYOUT_BM_7KEY_A = 24;
+    BUTTON_LAYOUT_GITADORA_FRET_A = 25;
+    BUTTON_LAYOUT_GITADORA_STRUM_A = 26;
+    BUTTON_LAYOUT_BOARD_DEFINED_A = 27;
+    BUTTON_LAYOUT_BANDHERO_FRET_A = 28;
+    BUTTON_LAYOUT_BANDHERO_STRUM_A = 29;
+    BUTTON_LAYOUT_6GAWD_A = 30;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTON_A = 31;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTONPLUS_A = 32;
+    BUTTON_LAYOUT_STICKLESS_R16 = 33;
+}
+
+enum ButtonLayoutRight
+{
+    option (nanopb_enumopt).long_names = false;
+
+    BUTTON_LAYOUT_ARCADE = 0;
+    BUTTON_LAYOUT_STICKLESSB = 1;
+    BUTTON_LAYOUT_BUTTONS_ANGLEDB = 2;
+    BUTTON_LAYOUT_VEWLIX = 3;
+    BUTTON_LAYOUT_VEWLIX7 = 4;
+    BUTTON_LAYOUT_CAPCOM = 5;
+    BUTTON_LAYOUT_CAPCOM6 = 6;
+    BUTTON_LAYOUT_SEGA2P = 7;
+    BUTTON_LAYOUT_NOIR8 = 8;
+    BUTTON_LAYOUT_KEYBOARDB = 9;
+    BUTTON_LAYOUT_DANCEPADB = 10;
+    BUTTON_LAYOUT_TWINSTICKB = 11;
+    BUTTON_LAYOUT_BLANKB = 12;
+    BUTTON_LAYOUT_VLXB = 13;
+    BUTTON_LAYOUT_FIGHTBOARD = 14;
+    BUTTON_LAYOUT_FIGHTBOARD_STICK_MIRRORED = 15;
+    BUTTON_LAYOUT_CUSTOMB = 16;
+    BUTTON_LAYOUT_KEYBOARD8B = 17;
+    BUTTON_LAYOUT_OPENCORE0WASDB = 18;
+    BUTTON_LAYOUT_STICKLESS_13B = 19;
+    BUTTON_LAYOUT_STICKLESS_16B = 20;
+    BUTTON_LAYOUT_STICKLESS_14B = 21;
+    BUTTON_LAYOUT_DANCEPAD_DDR_RIGHT = 22;
+    BUTTON_LAYOUT_DANCEPAD_PIU_RIGHT = 23;
+    BUTTON_LAYOUT_POPN_B = 24;
+    BUTTON_LAYOUT_TAIKO_B = 25;
+    BUTTON_LAYOUT_BM_TURNTABLE_B = 26;
+    BUTTON_LAYOUT_BM_5KEY_B = 27;
+    BUTTON_LAYOUT_BM_7KEY_B = 28;
+    BUTTON_LAYOUT_GITADORA_FRET_B = 29;
+    BUTTON_LAYOUT_GITADORA_STRUM_B = 30;
+    BUTTON_LAYOUT_BOARD_DEFINED_B = 31;
+    BUTTON_LAYOUT_BANDHERO_FRET_B = 32;
+    BUTTON_LAYOUT_BANDHERO_STRUM_B = 33;
+    BUTTON_LAYOUT_6GAWD_B = 34;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTON_B = 35;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTONPLUS_B = 36;
+    BUTTON_LAYOUT_STICKLESS_R16B = 37;
+}
+
+enum SplashMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    SPLASH_MODE_STATIC = 0;
+    SPLASH_MODE_CLOSEIN = 1;
+    SPLASH_MODE_CLOSEINCUSTOM = 2;
+    SPLASH_MODE_NONE = 3;
+}
+
+enum SplashChoice
+{
+    option (nanopb_enumopt).long_names = false;
+
+    SPLASH_CHOICE_MAIN = 0;
+    SPLASH_CHOICE_X = 1;
+    SPLASH_CHOICE_Y = 2;
+    SPLASH_CHOICE_Z = 3;
+    SPLASH_CHOICE_CUSTOM = 4;
+    SPLASH_CHOICE_LEGACY = 5;
+}
+
+enum OnBoardLedMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    ON_BOARD_LED_MODE_OFF = 0;
+    ON_BOARD_LED_MODE_MODE_INDICATOR = 1;
+    ON_BOARD_LED_MODE_INPUT_TEST = 2;
+    ON_BOARD_LED_MODE_PS_AUTH = 3;
+}
+
+enum InputMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    INPUT_MODE_XINPUT = 0;
+    INPUT_MODE_SWITCH = 1;
+    INPUT_MODE_PS3 = 2;
+    INPUT_MODE_KEYBOARD = 3;
+    INPUT_MODE_PS4 = 4;
+    INPUT_MODE_XBONE = 5;
+    INPUT_MODE_MDMINI = 6;
+    INPUT_MODE_NEOGEO = 7;
+    INPUT_MODE_PCEMINI = 8;
+    INPUT_MODE_EGRET = 9;
+    INPUT_MODE_ASTRO = 10;
+    INPUT_MODE_PSCLASSIC = 11;
+    INPUT_MODE_XBOXORIGINAL = 12;
+    INPUT_MODE_PS5 = 13;
+    INPUT_MODE_GENERIC = 14;
+    INPUT_MODE_CONFIG = 255;
+}
+
+enum InputModeAuthType
+{
+    option (nanopb_enumopt).long_names = false;
+
+    INPUT_MODE_AUTH_TYPE_NONE = 0;
+    INPUT_MODE_AUTH_TYPE_KEYS = 1;
+    INPUT_MODE_AUTH_TYPE_USB = 2;
+    INPUT_MODE_AUTH_TYPE_I2C = 3;
+}
+
+enum DpadMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    DPAD_MODE_DIGITAL = 0;
+    DPAD_MODE_LEFT_ANALOG = 1;
+    DPAD_MODE_RIGHT_ANALOG = 2;
+}
+
+enum InvertMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    INVERT_NONE = 0;
+    INVERT_X = 1;
+    INVERT_Y = 2;
+    INVERT_XY = 3;
+}
+
+enum SOCDMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    SOCD_MODE_UP_PRIORITY = 0;				// U+D=U, L+R=N
+    SOCD_MODE_NEUTRAL = 1;					// U+D=N, L+R=N
+    SOCD_MODE_SECOND_INPUT_PRIORITY = 2;	// U>D=D, L>R=R (Last Input Priority, aka Last Win)
+    SOCD_MODE_FIRST_INPUT_PRIORITY = 3;		// U>D=U, L>R=L (First Input Priority, aka First Win)
+    SOCD_MODE_BYPASS = 4;					// U+D=UD, L+R=LR (No cleaning applied)
+}
+
+enum GpioAction
+{
+    option (nanopb_enumopt).long_names = false;
+
+    // the lowest value is the default, which should be NONE;
+    // reserving some numbers in case we need more not-mapped type values
+    NONE                         = -10;
+    RESERVED                     = -5;
+    ASSIGNED_TO_ADDON            = 0;
+    BUTTON_PRESS_UP              = 1;
+    BUTTON_PRESS_DOWN            = 2;
+    BUTTON_PRESS_LEFT            = 3;
+    BUTTON_PRESS_RIGHT           = 4;
+    BUTTON_PRESS_B1              = 5;
+    BUTTON_PRESS_B2              = 6;
+    BUTTON_PRESS_B3              = 7;
+    BUTTON_PRESS_B4              = 8;
+    BUTTON_PRESS_L1              = 9;
+    BUTTON_PRESS_R1              = 10;
+    BUTTON_PRESS_L2              = 11;
+    BUTTON_PRESS_R2              = 12;
+    BUTTON_PRESS_S1              = 13;
+    BUTTON_PRESS_S2              = 14;
+    BUTTON_PRESS_A1              = 15;
+    BUTTON_PRESS_A2              = 16;
+    BUTTON_PRESS_L3              = 17;
+    BUTTON_PRESS_R3              = 18;
+    BUTTON_PRESS_FN              = 19;
+    BUTTON_PRESS_DDI_UP          = 20;
+    BUTTON_PRESS_DDI_DOWN        = 21;
+    BUTTON_PRESS_DDI_LEFT        = 22;
+    BUTTON_PRESS_DDI_RIGHT       = 23;
+    SUSTAIN_DP_MODE_DP           = 24;
+    SUSTAIN_DP_MODE_LS           = 25;
+    SUSTAIN_DP_MODE_RS           = 26;
+    SUSTAIN_SOCD_MODE_UP_PRIO    = 27;
+    SUSTAIN_SOCD_MODE_NEUTRAL    = 28;
+    SUSTAIN_SOCD_MODE_SECOND_WIN = 29;
+    SUSTAIN_SOCD_MODE_FIRST_WIN  = 30;
+    SUSTAIN_SOCD_MODE_BYPASS     = 31;
+    BUTTON_PRESS_TURBO           = 32;
+    BUTTON_PRESS_MACRO           = 33;
+    BUTTON_PRESS_MACRO_1         = 34;
+    BUTTON_PRESS_MACRO_2         = 35;
+    BUTTON_PRESS_MACRO_3         = 36;
+    BUTTON_PRESS_MACRO_4         = 37;
+    BUTTON_PRESS_MACRO_5         = 38;
+    BUTTON_PRESS_MACRO_6         = 39;
+    CUSTOM_BUTTON_COMBO          = 40;
+    BUTTON_PRESS_A3              = 41;
+    BUTTON_PRESS_A4              = 42;
+    BUTTON_PRESS_E1              = 43;
+    BUTTON_PRESS_E2              = 44;
+    BUTTON_PRESS_E3              = 45;
+    BUTTON_PRESS_E4              = 46;
+    BUTTON_PRESS_E5              = 47;
+    BUTTON_PRESS_E6              = 48;
+    BUTTON_PRESS_E7              = 49;
+    BUTTON_PRESS_E8              = 50;
+    BUTTON_PRESS_E9              = 51;
+    BUTTON_PRESS_E10             = 52;
+    BUTTON_PRESS_E11             = 53;
+    BUTTON_PRESS_E12             = 54;
+}
+
+enum GpioDirection
+{
+    option (nanopb_enumopt).long_names = false;
+
+    GPIO_DIRECTION_INPUT = 0;
+    GPIO_DIRECTION_OUTPUT = 1;
+}
+
+enum GamepadHotkey
+{
+    option (nanopb_enumopt).long_names = false;
+
+    HOTKEY_NONE                  = 0;
+    HOTKEY_DPAD_DIGITAL          = 1;
+    HOTKEY_DPAD_LEFT_ANALOG      = 2;
+    HOTKEY_DPAD_RIGHT_ANALOG     = 3;
+    HOTKEY_HOME_BUTTON           = 4;
+    HOTKEY_CAPTURE_BUTTON        = 5;
+    HOTKEY_SOCD_UP_PRIORITY      = 6;
+    HOTKEY_SOCD_NEUTRAL          = 7;
+    HOTKEY_SOCD_LAST_INPUT       = 8;
+    HOTKEY_INVERT_X_AXIS         = 9;
+    HOTKEY_INVERT_Y_AXIS         = 10;
+    HOTKEY_SOCD_FIRST_INPUT      = 11;
+    HOTKEY_SOCD_BYPASS           = 12;
+    HOTKEY_TOGGLE_4_WAY_MODE     = 13;
+    HOTKEY_TOGGLE_DDI_4_WAY_MODE = 14;
+    HOTKEY_LOAD_PROFILE_1        = 15;
+    HOTKEY_LOAD_PROFILE_2        = 16;
+    HOTKEY_LOAD_PROFILE_3        = 17;
+    HOTKEY_LOAD_PROFILE_4        = 18;
+    HOTKEY_L3_BUTTON             = 19;
+    HOTKEY_R3_BUTTON             = 20;
+    HOTKEY_TOUCHPAD_BUTTON       = 21;
+    HOTKEY_REBOOT_DEFAULT        = 22;
+    HOTKEY_B1_BUTTON             = 23;
+    HOTKEY_B2_BUTTON             = 24;
+    HOTKEY_B3_BUTTON             = 25;
+    HOTKEY_B4_BUTTON             = 26;
+    HOTKEY_L1_BUTTON             = 27;
+    HOTKEY_R1_BUTTON             = 28;
+    HOTKEY_L2_BUTTON             = 29;
+    HOTKEY_R2_BUTTON             = 30;
+    HOTKEY_S1_BUTTON             = 31;
+    HOTKEY_S2_BUTTON             = 32;
+    HOTKEY_A1_BUTTON             = 33;
+    HOTKEY_A2_BUTTON             = 34;
+    HOTKEY_NEXT_PROFILE          = 35;
+    HOTKEY_A3_BUTTON             = 36;
+    HOTKEY_A4_BUTTON             = 37;
+    HOTKEY_DPAD_UP               = 38;
+    HOTKEY_DPAD_DOWN             = 39;
+    HOTKEY_DPAD_LEFT             = 40;
+    HOTKEY_DPAD_RIGHT            = 41;
+    HOTKEY_PREVIOUS_PROFILE      = 42;
+}
+
+// This has to be kept in sync with LEDFormat in NeoPico.hpp
+enum LEDFormat_Proto
+{
+    LED_FORMAT_GRB = 0;
+    LED_FORMAT_RGB = 1;
+    LED_FORMAT_GRBW = 2;
+    LED_FORMAT_RGBW = 3;
+}
+
+enum ShmupMixMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    SHMUP_MIX_MODE_TURBO_PRIORITY = 0;
+    SHMUP_MIX_MODE_CHARGE_PRIORITY = 1;
+}
+
+enum PLEDType
+{
+    option (nanopb_enumopt).long_names = false;
+
+    PLED_TYPE_NONE = -1;
+    PLED_TYPE_PWM = 0;
+    PLED_TYPE_RGB = 1;
+};
+
+enum ForcedSetupMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+	FORCED_SETUP_MODE_OFF = 0;
+	FORCED_SETUP_MODE_LOCK_MODE_SWITCH = 1;
+	FORCED_SETUP_MODE_LOCK_WEB_CONFIG = 2;
+	FORCED_SETUP_MODE_LOCK_BOTH = 3;
+};
+
+enum DualDirectionalCombinationMode
+{
+	option (nanopb_enumopt).long_names = false;
+
+	MIXED_MODE = 0;
+	GAMEPAD_MODE = 1;
+	DUAL_MODE = 2;
+	NONE_MODE = 3;
+}
+
+enum PS4ControllerType
+{
+    option (nanopb_enumopt).long_names = false;
+
+    PS4_CONTROLLER = 0;
+    PS4_ARCADESTICK = 7;
+}
+
+enum MacroType
+{
+    option (nanopb_enumopt).long_names = false;
+
+    ON_PRESS = 1;
+    ON_HOLD_REPEAT = 2;
+    ON_TOGGLE = 3;
+};
+
+enum GPElement
+{
+    option (nanopb_enumopt).long_names = false;
+
+    GP_ELEMENT_WIDGET = 0;
+    GP_ELEMENT_SCREEN = 1;
+    GP_ELEMENT_BTN_BUTTON = 2;
+    GP_ELEMENT_DIR_BUTTON = 3;
+    GP_ELEMENT_PIN_BUTTON = 4;
+    GP_ELEMENT_LEVER = 5;
+    GP_ELEMENT_LABEL = 6;
+    GP_ELEMENT_SPRITE = 7;
+    GP_ELEMENT_SHAPE = 8;
+};
+
+enum GPShape_Type
+{
+    option (nanopb_enumopt).long_names = false;
+
+    GP_SHAPE_ELLIPSE = 0;
+    GP_SHAPE_SQUARE = 1;
+    GP_SHAPE_LINE = 2;
+    GP_SHAPE_POLYGON = 3;
+    GP_SHAPE_ARC = 4;
+};
+
+enum RotaryEncoderPinMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    ENCODER_MODE_NONE = 0;
+    ENCODER_MODE_LEFT_ANALOG_X = 1;
+    ENCODER_MODE_LEFT_ANALOG_Y = 2;
+    ENCODER_MODE_RIGHT_ANALOG_X = 3;
+    ENCODER_MODE_RIGHT_ANALOG_Y = 4;
+    ENCODER_MODE_LEFT_TRIGGER = 5;
+    ENCODER_MODE_RIGHT_TRIGGER = 6;
+    ENCODER_MODE_DPAD_X = 7;
+    ENCODER_MODE_DPAD_Y = 8;
+};
+
+enum ReactiveLEDMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    REACTIVE_LED_STATIC_OFF = 0;
+    REACTIVE_LED_STATIC_ON = 1;
+    REACTIVE_LED_FADE_IN = 2;
+    REACTIVE_LED_FADE_OUT = 3;
+};
+
+enum PS4ControllerIDMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    PS4_ID_CONSOLE = 0;
+    PS4_ID_EMULATION = 1;
+};

gp2040ce_bintools/proto_snapshot/nanopb.proto

@@ -0,0 +1,189 @@
+// Custom options for defining:
+// - Maximum size of string/bytes
+// - Maximum number of elements in array
+//
+// These are used by nanopb to generate statically allocable structures
+// for memory-limited environments.
+
+syntax = "proto2";
+import "google/protobuf/descriptor.proto";
+
+option java_package = "fi.kapsi.koti.jpa.nanopb";
+
+enum FieldType {
+    FT_DEFAULT = 0; // Automatically decide field type, generate static field if possible.
+    FT_CALLBACK = 1; // Always generate a callback field.
+    FT_POINTER = 4; // Always generate a dynamically allocated field.
+    FT_STATIC = 2; // Generate a static field or raise an exception if not possible.
+    FT_IGNORE = 3; // Ignore the field completely.
+    FT_INLINE = 5; // Legacy option, use the separate 'fixed_length' option instead
+}
+
+enum IntSize {
+    IS_DEFAULT = 0; // Default, 32/64bit based on type in .proto
+    IS_8 = 8;
+    IS_16 = 16;
+    IS_32 = 32;
+    IS_64 = 64;
+}
+
+enum TypenameMangling {
+    M_NONE = 0; // Default, no typename mangling
+    M_STRIP_PACKAGE = 1; // Strip current package name
+    M_FLATTEN = 2; // Only use last path component
+    M_PACKAGE_INITIALS = 3; // Replace the package name by the initials
+}
+
+enum DescriptorSize {
+    DS_AUTO = 0; // Select minimal size based on field type
+    DS_1 = 1;    // 1 word; up to 15 byte fields, no arrays
+    DS_2 = 2;    // 2 words; up to 4095 byte fields, 4095 entry arrays
+    DS_4 = 4;    // 4 words; up to 2^32-1 byte fields, 2^16-1 entry arrays
+    DS_8 = 8;    // 8 words; up to 2^32-1 entry arrays
+}
+
+// This is the inner options message, which basically defines options for
+// a field. When it is used in message or file scope, it applies to all
+// fields.
+message NanoPBOptions {
+  // Allocated size for 'bytes' and 'string' fields.
+  // For string fields, this should include the space for null terminator.
+  optional int32 max_size = 1;
+  
+  // Maximum length for 'string' fields. Setting this is equivalent
+  // to setting max_size to a value of length+1.
+  optional int32 max_length = 14;
+  
+  // Allocated number of entries in arrays ('repeated' fields)
+  optional int32 max_count = 2;
+  
+  // Size of integer fields. Can save some memory if you don't need
+  // full 32 bits for the value.
+  optional IntSize int_size = 7 [default = IS_DEFAULT];
+
+  // Force type of field (callback or static allocation)
+  optional FieldType type = 3 [default = FT_DEFAULT];
+  
+  // Use long names for enums, i.e. EnumName_EnumValue.
+  optional bool long_names = 4 [default = true];
+  
+  // Add 'packed' attribute to generated structs.
+  // Note: this cannot be used on CPUs that break on unaligned
+  // accesses to variables.
+  optional bool packed_struct = 5 [default = false];
+  
+  // Add 'packed' attribute to generated enums.
+  optional bool packed_enum = 10 [default = false];
+  
+  // Skip this message
+  optional bool skip_message = 6 [default = false];
+
+  // Generate oneof fields as normal optional fields instead of union.
+  optional bool no_unions = 8 [default = false];
+
+  // integer type tag for a message
+  optional uint32 msgid = 9;
+
+  // decode oneof as anonymous union
+  optional bool anonymous_oneof = 11 [default = false];
+
+  // Proto3 singular field does not generate a "has_" flag
+  optional bool proto3 = 12 [default = false];
+  
+  // Force proto3 messages to have no "has_" flag.
+  // This was default behavior until nanopb-0.4.0.
+  optional bool proto3_singular_msgs = 21 [default = false];
+
+  // Generate an enum->string mapping function (can take up lots of space).
+  optional bool enum_to_string = 13 [default = false];
+
+  // Generate bytes arrays with fixed length
+  optional bool fixed_length = 15 [default = false];
+
+  // Generate repeated field with fixed count
+  optional bool fixed_count = 16 [default = false];
+
+  // Generate message-level callback that is called before decoding submessages.
+  // This can be used to set callback fields for submsgs inside oneofs.
+  optional bool submsg_callback = 22 [default = false];
+
+  // Shorten or remove package names from type names.
+  // This option applies only on the file level.
+  optional TypenameMangling mangle_names = 17 [default = M_NONE];
+
+  // Data type for storage associated with callback fields.
+  optional string callback_datatype = 18 [default = "pb_callback_t"];
+
+  // Callback function used for encoding and decoding.
+  // Prior to nanopb-0.4.0, the callback was specified in per-field pb_callback_t
+  // structure. This is still supported, but does not work inside e.g. oneof or pointer
+  // fields. Instead, a new method allows specifying a per-message callback that
+  // will be called for all callback fields in a message type.
+  optional string callback_function = 19 [default = "pb_default_field_callback"];
+
+  // Select the size of field descriptors. This option has to be defined
+  // for the whole message, not per-field. Usually automatic selection is
+  // ok, but if it results in compilation errors you can increase the field
+  // size here.
+  optional DescriptorSize descriptorsize = 20 [default = DS_AUTO];
+
+  // Set default value for has_ fields.
+  optional bool default_has = 23 [default = false];
+
+  // Extra files to include in generated `.pb.h`
+  repeated string include = 24;
+
+  // Automatic includes to exclude from generated `.pb.h`
+  // Same as nanopb_generator.py command line flag -x.
+  repeated string exclude = 26;
+
+  // Package name that applies only for nanopb.
+  optional string package = 25;
+  
+  // Override type of the field in generated C code. Only to be used with related field types
+  optional google.protobuf.FieldDescriptorProto.Type type_override = 27;
+
+  // Due to historical reasons, nanopb orders fields in structs by their tag number
+  // instead of the order in .proto. Set this to false to keep the .proto order.
+  // The default value will probably change to false in nanopb-0.5.0.
+  optional bool sort_by_tag = 28 [default = true];
+
+  // Set the FT_DEFAULT field conversion strategy.
+  // A field that can become a static member of a c struct (e.g. int, bool, etc)
+  // will be a a static field.
+  // Fields with dynamic length are converted to either a pointer or a callback.
+  optional FieldType fallback_type = 29 [default = FT_CALLBACK]; 
+
+  // GP2040-CE extension
+  // Marks a field to be excluded when performing export operations (i.e. converting to JSON)
+  optional bool disallow_export = 30 [default = false];
+}
+
+// Extensions to protoc 'Descriptor' type in order to define options
+// inside a .proto file.
+//
+// Protocol Buffers extension number registry
+// --------------------------------
+// Project:  Nanopb
+// Contact:  Petteri Aimonen <jpa@kapsi.fi>
+// Web site: http://kapsi.fi/~jpa/nanopb
+// Extensions: 1010 (all types)
+// --------------------------------
+
+extend google.protobuf.FileOptions {
+    optional NanoPBOptions nanopb_fileopt = 1010;
+}
+
+extend google.protobuf.MessageOptions {
+    optional NanoPBOptions nanopb_msgopt = 1010;
+}
+
+extend google.protobuf.EnumOptions {
+    optional NanoPBOptions nanopb_enumopt = 1010;
+}
+
+extend google.protobuf.FieldOptions {
+    optional NanoPBOptions nanopb = 1010;
+}
+
+

gp2040ce_bintools/rp2040.py

@@ -0,0 +1,242 @@
+"""Methods to interact with the Raspberry Pi RP2040 directly.
+
+Much of this code is a partial Python implementation of picotool.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
+import logging
+import struct
+
+import usb.core
+
+logger = logging.getLogger(__name__)
+
+PICO_VENDOR = 0x2e8a
+PICO_PRODUCT = 0x0003
+
+PICOBOOT_CMD_STRUCT = '<LLBBxxL'
+PICOBOOT_CMD_ERASE_SUFFIX_STRUCT = 'LL8x'
+PICOBOOT_CMD_EXCLUSIVE_ACCESS_SUFFIX_STRUCT = 'L12x'
+PICOBOOT_CMD_EXIT_XIP_SUFFIX_STRUCT = '16x'
+PICOBOOT_CMD_READ_SUFFIX_STRUCT = 'LL8x'
+PICOBOOT_CMD_REBOOT_SUFFIX_STRUCT = 'LLL4x'
+
+PICO_MAGIC = 0x431fd10b
+PICO_SRAM_END = 0x20042000
+# only a partial implementation...
+PICO_COMMANDS = {
+    'EXCLUSIVE_ACCESS': 0x1,
+    'REBOOT': 0x2,
+    'ERASE': 0x3,
+    'READ': 0x4,
+    'WRITE': 0x5,
+    'EXIT_XIP': 0x6,
+}
+
+
+#################
+# LIBRARY ITEMS #
+#################
+
+
+class RP2040AlignmentError(ValueError):
+    """Exception raised when the address provided for an operation is invalid."""
+
+
+def get_bootsel_endpoints() -> tuple[usb.core.Endpoint, usb.core.Endpoint]:
+    """Retrieve the USB endpoint for purposes of interacting with a RP2040 in BOOTSEL mode.
+
+    Returns:
+        the out and in endpoints for the BOOTSEL interface
+    """
+    # get the device and claim it from whatever else might have in the kernel
+    pico_device = usb.core.find(idVendor=PICO_VENDOR, idProduct=PICO_PRODUCT)
+
+    if not pico_device:
+        raise ValueError("RP2040 board in BOOTSEL mode could not be found!")
+
+    try:
+        if pico_device.is_kernel_driver_active(0):
+            pico_device.detach_kernel_driver(0)
+    except NotImplementedError:
+        # detaching the driver is for *nix, not possible/relevant on Windows
+        pass
+
+    pico_configuration = pico_device.get_active_configuration()
+    # two interfaces are present, we want the direct rather than mass storage
+    # pico_bootsel_interface = pico_configuration[(1, 0)]
+    pico_bootsel_interface = usb.util.find_descriptor(pico_configuration,
+                                                      custom_match=lambda e: e.bInterfaceClass == 0xff)
+    out_endpoint = usb.util.find_descriptor(pico_bootsel_interface,
+                                            custom_match=lambda e: (usb.util.endpoint_direction(e.bEndpointAddress) ==
+                                                                    usb.util.ENDPOINT_OUT))
+    in_endpoint = usb.util.find_descriptor(pico_bootsel_interface,
+                                           custom_match=lambda e: (usb.util.endpoint_direction(e.bEndpointAddress) ==
+                                                                   usb.util.ENDPOINT_IN))
+    return out_endpoint, in_endpoint
+
+
+def exclusive_access(out_end: usb.core.Endpoint, in_end: usb.core.Endpoint, is_exclusive: bool = True) -> None:
+    """Enable exclusive access mode on a RP2040 in BOOTSEL.
+
+    Args:
+        out_endpoint: the out direction USB endpoint to write to
+        in_endpoint: the in direction USB endpoint to read from
+    """
+    # set up the data
+    pico_token = 1
+    command_size = 1
+    transfer_len = 0
+    exclusive = 1 if is_exclusive else 0
+    payload = struct.pack(PICOBOOT_CMD_STRUCT + PICOBOOT_CMD_EXCLUSIVE_ACCESS_SUFFIX_STRUCT,
+                          PICO_MAGIC, pico_token, PICO_COMMANDS['EXCLUSIVE_ACCESS'], command_size, transfer_len,
+                          exclusive)
+    logger.debug("EXCLUSIVE_ACCESS: %s", payload)
+    out_end.write(payload)
+    _ = in_end.read(256)
+
+
+def erase(out_end: usb.core.Endpoint, in_end: usb.core.Endpoint, location: int, size: int) -> None:
+    """Erase a section of flash memory on a RP2040 in BOOTSEL mode.
+
+    Args:
+        out_endpoint: the out direction USB endpoint to write to
+        in_endpoint: the in direction USB endpoint to read from
+        location: memory address of where to start erasing from
+        size: number of bytes to erase
+    """
+    logger.debug("clearing %s bytes starting at %s", size, hex(location))
+    # set up the data
+    pico_token = 1
+    command_size = 8
+    transfer_len = 0
+    payload = struct.pack(PICOBOOT_CMD_STRUCT + PICOBOOT_CMD_ERASE_SUFFIX_STRUCT,
+                          PICO_MAGIC, pico_token, PICO_COMMANDS['ERASE'], command_size, transfer_len,
+                          location, size)
+    logger.debug("ERASE: %s", payload)
+    out_end.write(payload)
+    _ = in_end.read(256)
+
+
+def exit_xip(out_end: usb.core.Endpoint, in_end: usb.core.Endpoint) -> None:
+    """Exit XIP on a RP2040 in BOOTSEL.
+
+    Args:
+        out_endpoint: the out direction USB endpoint to write to
+        in_endpoint: the in direction USB endpoint to read from
+    """
+    # set up the data
+    pico_token = 1
+    command_size = 0
+    transfer_len = 0
+    payload = struct.pack(PICOBOOT_CMD_STRUCT + PICOBOOT_CMD_EXIT_XIP_SUFFIX_STRUCT,
+                          PICO_MAGIC, pico_token, PICO_COMMANDS['EXIT_XIP'], command_size, transfer_len)
+    logger.debug("EXIT_XIP: %s", payload)
+    out_end.write(payload)
+    _ = in_end.read(256)
+
+
+def read(out_end: usb.core.Endpoint, in_end: usb.core.Endpoint, location: int, size: int) -> bytearray:
+    """Read a requested number of bytes from a RP2040 in BOOTSEL, starting from the specified location.
+
+    This also prepares the USB device for reading, so it expects to be able to grab
+    exclusive access.
+
+    Args:
+        out_endpoint: the out direction USB endpoint to write to
+        in_endpoint: the in direction USB endpoint to read from
+        location: memory address of where to start reading from
+        size: number of bytes to read
+    Returns:
+        the read bytes as a byte array
+    """
+    # set up the data
+    chunk_size = 256
+    command_size = 8
+
+    read_location = location
+    read_size = 0
+    content = bytearray()
+    exclusive_access(out_end, in_end, is_exclusive=True)
+    while read_size < size:
+        exit_xip(out_end, in_end)
+        pico_token = 1
+        payload = struct.pack(PICOBOOT_CMD_STRUCT + PICOBOOT_CMD_READ_SUFFIX_STRUCT,
+                              PICO_MAGIC, pico_token, PICO_COMMANDS['READ'] + 128, command_size, chunk_size,
+                              read_location, chunk_size)
+        logger.debug("READ: %s", payload)
+        out_end.write(payload)
+        res = in_end.read(chunk_size)
+        logger.debug("res: %s", res)
+        content += res
+        read_size += chunk_size
+        read_location += chunk_size
+        out_end.write(b'\xc0')
+    exclusive_access(out_end, in_end, is_exclusive=False)
+    logger.debug("final content: %s", content[:size])
+    return content[:size]
+
+
+def reboot(out_end: usb.core.Endpoint) -> None:
+    """Reboot a RP2040 in BOOTSEL mode."""
+    # set up the data
+    pico_token = 1
+    command_size = 12
+    transfer_len = 0
+    boot_start = 0
+    boot_end = PICO_SRAM_END
+    boot_delay_ms = 500
+    out_end.write(struct.pack(PICOBOOT_CMD_STRUCT + PICOBOOT_CMD_REBOOT_SUFFIX_STRUCT,
+                              PICO_MAGIC, pico_token, PICO_COMMANDS['REBOOT'], command_size, transfer_len,
+                              boot_start, boot_end, boot_delay_ms))
+    # we don't even bother reading here because it may have already rebooted
+
+
+def write(out_end: usb.core.Endpoint, in_end: usb.core.Endpoint, location: int, content: bytes) -> None:
+    """Write content to a RP2040 in BOOTSEL, starting from the specified location.
+
+    This also prepares the USB device for writing, so it expects to be able to grab
+    exclusive access.
+
+    Args:
+        out_endpoint: the out direction USB endpoint to write to
+        in_endpoint: the in direction USB endpoint to read from
+        location: memory address of where to start reading from
+        content: the data to write
+    """
+    chunk_size = 4096
+    write_location = location
+    write_size = 0
+
+    if (location % chunk_size) != 0:
+        raise RP2040AlignmentError(f"writes must start at {chunk_size} byte boundaries, "
+                                   f"please pad or align as appropriate!")
+
+    # set up the data
+    command_size = 8
+    size = len(content)
+
+    exclusive_access(out_end, in_end, is_exclusive=True)
+    while write_size < size:
+        pico_token = 1
+        to_write = content[write_size:(write_size + chunk_size)]
+
+        exit_xip(out_end, in_end)
+        logger.debug("erasing %s bytes at %s", len(to_write), hex(write_location))
+        erase(out_end, in_end, write_location, len(to_write))
+
+        logger.debug("writing %s bytes to %s", len(to_write), hex(write_location))
+        payload = struct.pack(PICOBOOT_CMD_STRUCT + PICOBOOT_CMD_READ_SUFFIX_STRUCT,
+                              PICO_MAGIC, pico_token, PICO_COMMANDS['WRITE'], command_size, len(to_write),
+                              write_location, len(to_write))
+        logger.debug("WRITE: %s", payload)
+        out_end.write(payload)
+        logger.debug("actually writing bytes now...")
+        logger.debug("payload: %s", to_write)
+        out_end.write(bytes(to_write))
+        res = in_end.read(chunk_size)
+        logger.debug("res: %s", res)
+        write_size += chunk_size
+        write_location += chunk_size
+    exclusive_access(out_end, in_end, is_exclusive=False)

gp2040ce_bintools/storage.py

@@ -1,34 +1,137 @@
-"""Interact with the protobuf config from a picotool flash dump of a GP2040-CE board."""
+"""Interact with the protobuf config from a picotool flash dump of a GP2040-CE board.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
 import argparse
+import binascii
 import logging
+import struct
 
 from google.protobuf.json_format import MessageToJson
+from google.protobuf.json_format import Parse as JsonParse
 from google.protobuf.message import Message
 
 from gp2040ce_bintools import core_parser, get_config_pb2
+from gp2040ce_bintools.rp2040 import get_bootsel_endpoints, read
 
 logger = logging.getLogger(__name__)
 
-STORAGE_LOCATION = 0x1FE000
-STORAGE_SIZE = 8192
+STORAGE_SIZE = 32768
+
+BOARD_CONFIG_BINARY_LOCATION = (2 * 1024 * 1024) - (STORAGE_SIZE * 2)   # 0x1F0000
+BOARD_CONFIG_BOOTSEL_ADDRESS = 0x10000000 + BOARD_CONFIG_BINARY_LOCATION
+USER_CONFIG_BINARY_LOCATION = (2 * 1024 * 1024) - STORAGE_SIZE          # 0x1F8000
+USER_CONFIG_BOOTSEL_ADDRESS = 0x10000000 + USER_CONFIG_BINARY_LOCATION
 
 FOOTER_SIZE = 12
 FOOTER_MAGIC = b'\x65\xe3\xf1\xd2'
 
+UF2_FAMILY_ID = 0xE48BFF56
+UF2_MAGIC_FIRST = 0x0A324655
+UF2_MAGIC_SECOND = 0x9E5D5157
+UF2_MAGIC_FINAL = 0x0AB16F30
+
 
 #################
 # LIBRARY ITEMS #
 #################
 
 
-class ConfigLengthError(ValueError):
+class ConfigReadError(ValueError):
+    """General exception for failing to read/verify the GP2040-CE config for some reason."""
+
+
+class ConfigCrcError(ConfigReadError):
+    """Exception raised when the CRC checksum in the footer doesn't match the actual content's."""
+
+
+class ConfigLengthError(ConfigReadError):
     """Exception raised when a length sanity check fails."""
 
 
-class ConfigMagicError(ValueError):
+class ConfigMagicError(ConfigReadError):
     """Exception raised when the config section does not have the magic value in its footer."""
 
 
+def convert_binary_to_uf2(binaries: list[tuple[int, bytearray]]) -> bytearray:
+    """Convert a GP2040-CE binary payload to Microsoft's UF2 format.
+
+    https://github.com/microsoft/uf2/tree/master#overview
+
+    Args:
+        binaries: list of start,binary pairs of binary data to write at the specified memory offset in flash
+    Returns:
+        the content in UF2 format
+    """
+    total_blocks = sum([(len(binary) // 256) + 1 if len(binary) % 256 else len(binary) // 256
+                        for offset, binary in binaries])
+    block_count = 0
+
+    uf2 = bytearray()
+    for start, binary in binaries:
+        size = len(binary)
+        index = 0
+        while index < size:
+            pad_count = 476 - len(binary[index:index+256])
+            uf2 += struct.pack('<LLLLLLLL',
+                               UF2_MAGIC_FIRST,                                 # first magic number
+                               UF2_MAGIC_SECOND,                                # second magic number
+                               0x00002000,                                      # familyID present
+                               0x10000000 + start + index,                      # address to write to
+                               256,                                             # bytes to write in this block
+                               block_count,                                     # sequential block number
+                               total_blocks,                                    # total number of blocks
+                               UF2_FAMILY_ID)                                   # family ID
+            uf2 += binary[index:index+256] + bytearray(b'\x00' * pad_count)     # content
+            uf2 += struct.pack('<L', UF2_MAGIC_FINAL)                           # final magic number
+            index += 256
+            block_count += 1
+    return uf2
+
+
+def convert_uf2_to_binary(uf2: bytearray) -> bytearray:
+    """Convert a Microsoft's UF2 payload to a raw binary.
+
+    https://github.com/microsoft/uf2/tree/master#overview
+
+    Args:
+        uf2: bytearray content to convert from a UF2 payload
+    Returns:
+        the content in sequential binary format
+    """
+    if len(uf2) % 512 != 0:
+        raise ValueError(f"provided binary is length {len(uf2)}, which isn't fully divisible by 512!")
+
+    binary = bytearray()
+    old_uf2_addr = None
+
+    for index in range(0, len(uf2), 512):
+        chunk = uf2[index:index+512]
+        _, _, _, uf2_addr, bytes_, block_num, block_count, _ = struct.unpack('<LLLLLLLL', chunk[0:32])
+        content = chunk[32:508]
+        if block_num != index // 512:
+            raise ValueError(f"inconsistent block number in reading UF2, got {block_num}, expected {index // 512}!")
+        if block_count != len(uf2) // 512:
+            raise ValueError(f"inconsistent block count in reading UF2, got {block_count}, expected {len(uf2) // 512}!")
+
+        if old_uf2_addr and (uf2_addr >= old_uf2_addr + bytes_):
+            # the new binary content is not immediately after what we wrote, it's further ahead, so pad
+            # the difference
+            binary += bytearray(b'\x00' * (uf2_addr - (old_uf2_addr + bytes_)))
+        elif old_uf2_addr and (uf2_addr < old_uf2_addr + bytes_):
+            # this is seeking backwards which we don't see yet
+            raise NotImplementedError("going backwards in binary files is not yet supported")
+
+        binary += content[0:bytes_]
+        old_uf2_addr = uf2_addr
+
+    # when this is all done we should have counted the expected number of blocks
+    if block_count != block_num + 1:
+        raise ValueError(f"not all expected blocks ({block_count}) were found, only got {block_num + 1}!")
+    return binary
+
+
 def get_config(content: bytes) -> Message:
     """Read the config from a GP2040-CE storage section.
 
@@ -46,20 +149,36 @@ def get_config(content: bytes) -> Message:
     return config
 
 
+def get_config_from_json(content: str) -> Message:
+    """Read the config represented by a JSON string.
+
+    Args:
+        content: JSON string representing a board config
+    Returns:
+        the parsed configuration
+    """
+    config_pb2 = get_config_pb2()
+    config = config_pb2.Config()
+    JsonParse(content, config)
+    logger.debug("parsed: %s", config)
+    return config
+
+
 def get_config_footer(content: bytes) -> tuple[int, int, str]:
     """Confirm and retrieve the config footer from a series of bytes of GP2040-CE storage.
 
     Args:
         content: bytes from a GP2040-CE board's storage section
     Returns:
-        the discovered config size, config CRC, and magic from the config footer
+        the discovered config size, config CRC checksum, and magic from the config footer
     Raises:
         ConfigLengthError, ConfigMagicError: if the provided bytes are not a config footer
     """
     # last 12 bytes are the footer
     logger.debug("length of content to look for footer in: %s", len(content))
+    logger.debug("content searching in for a footer: %s", content)
     if len(content) < FOOTER_SIZE:
-        raise ConfigLengthError("provided content is not large enough to have a config footer!")
+        raise ConfigLengthError(f"provided content ({len(content)} bytes) is not large enough to have a config footer!")
 
     footer = content[-FOOTER_SIZE:]
     logger.debug("suspected footer magic: %s", footer[-4:])
@@ -70,35 +189,135 @@ def get_config_footer(content: bytes) -> tuple[int, int, str]:
     config_crc = int.from_bytes(reversed(footer[4:8]), 'big')
     config_magic = f'0x{footer[8:12].hex()}'
 
-    # one last sanity check
+    # more sanity checks
+    logger.debug("length of content + footer: %s", len(content))
     if len(content) < config_size + FOOTER_SIZE:
-        raise ConfigLengthError("provided content is not large enough according to the config footer!")
+        raise ConfigLengthError(f"provided content ({len(content)} bytes) is not large enough according to the "
+                                f"config footer!")
+
+    logger.debug("config size according to footer: %s", config_size)
+
+    content_config = content[-(config_size + 12):-12]
+    content_crc = binascii.crc32(content_config)
+    logger.debug("content used to calculate CRC: %s", content_config)
+    logger.debug("calculated config CRC: %s", content_crc)
+    logger.debug("expected config CRC: %s", config_crc)
+    if config_crc != content_crc:
+        raise ConfigCrcError(f"provided content CRC checksum {content_crc} does not match footer's expected CRC "
+                             f"checksum {config_crc}!")
 
     logger.debug("detected footer (size:%s, crc:%s, magic:%s", config_size, config_crc, config_magic)
     return config_size, config_crc, config_magic
 
 
-def get_config_from_file(filename: str, whole_board: bool = False) -> Message:
+def get_binary_from_file(filename: str) -> bytes:
+    """Read the specified file (.bin or .uf2) and get back its raw binary contents.
+
+    Args:
+        filename: the filename of the file to open and read
+    Returns:
+        the file's content, in raw binary format
+    Raises:
+        FileNotFoundError: if the file was not found
+    """
+    with open(filename, 'rb') as dump:
+        if filename[-4:] == '.uf2':
+            content = bytes(convert_uf2_to_binary(bytearray(dump.read())))
+        else:
+            content = dump.read()
+
+    return content
+
+
+def get_config_from_file(filename: str, whole_board: bool = False, allow_no_file: bool = False,
+                         board_config: bool = False) -> Message:
     """Read the specified file (memory dump or whole board dump) and get back its config section.
 
     Args:
         filename: the filename of the file to open and read
         whole_board: optional, if true, attempt to find the storage section from its normal location on a board
+        allow_no_file: if true, attempting to open a nonexistent file returns an empty config, else it errors
+        board_config: if true, the board config is provided instead of the user config
     Returns:
         the parsed configuration
     """
-    with open(filename, 'rb') as dump:
-        content = dump.read()
-
-    if whole_board:
-        return get_config(get_storage_section(content))
-    else:
-        return get_config(content)
+    try:
+        if filename[-5:] == '.json':
+            with open(filename) as file_:
+                return get_config_from_json(file_.read())
+        else:
+            content = get_binary_from_file(filename)
+            if whole_board:
+                if board_config:
+                    return get_config(get_board_storage_section(content))
+                else:
+                    return get_config(get_user_storage_section(content))
+            else:
+                return get_config(content)
+    except FileNotFoundError:
+        if not allow_no_file:
+            raise
+        config_pb2 = get_config_pb2()
+        return config_pb2.Config()
 
 
-def get_storage_section(content: bytes) -> bytes:
+def get_config_from_usb(address: int) -> tuple[Message, object, object]:
+    """Read a config section from a USB device and provide the protobuf Message.
+
+    Args:
+        address: location of the flash to start reading from
+    Returns:
+        the parsed configuration, along with the USB out and in endpoints for reference
+    """
+    # open the USB device and get the config
+    endpoint_out, endpoint_in = get_bootsel_endpoints()
+    logger.debug("reading DEVICE ID %s:%s, bus %s, address %s", hex(endpoint_out.device.idVendor),
+                 hex(endpoint_out.device.idProduct), endpoint_out.device.bus, endpoint_out.device.address)
+    storage = read(endpoint_out, endpoint_in, address, STORAGE_SIZE)
+    return get_config(bytes(storage)), endpoint_out, endpoint_in
+
+
+def get_board_config_from_usb() -> tuple[Message, object, object]:
+    """Read the board configuration from the detected USB device.
+
+    Returns:
+        the parsed configuration, along with the USB out and in endpoints for reference
+    """
+    return get_config_from_usb(BOARD_CONFIG_BOOTSEL_ADDRESS)
+
+
+def get_user_config_from_usb() -> tuple[Message, object, object]:
+    """Read the user configuration from the detected USB device.
+
+    Returns:
+        the parsed configuration, along with the USB out and in endpoints for reference
+    """
+    return get_config_from_usb(USER_CONFIG_BOOTSEL_ADDRESS)
+
+
+def get_storage_section(content: bytes, address: int) -> bytes:
     """Pull out what should be the GP2040-CE storage section from a whole board dump.
 
+    Args:
+        content: bytes of a GP2040-CE whole board dump
+        address: location of the binary file to start reading from
+    Returns:
+        the presumed storage section from the binary
+    Raises:
+        ConfigLengthError: if the provided bytes don't appear to have a storage section
+    """
+    # a whole board must be at least as big as the known fences
+    logger.debug("length of content to look for storage in: %s", len(content))
+    if len(content) < address + STORAGE_SIZE:
+        raise ConfigLengthError("provided content is not large enough to have a storage section!")
+
+    logger.debug("returning bytes from %s to %s", hex(address), hex(address + STORAGE_SIZE))
+    return content[address:(address + STORAGE_SIZE)]
+
+
+def get_board_storage_section(content: bytes) -> bytes:
+    """Get the board storage area from what should be a whole board GP2040-CE dump.
+
     Args:
         content: bytes of a GP2040-CE whole board dump
     Returns:
@@ -106,13 +325,30 @@ def get_storage_section(content: bytes) -> bytes:
     Raises:
         ConfigLengthError: if the provided bytes don't appear to have a storage section
     """
-    # a whole board must be at least as big as the known fences
-    logger.debug("length of content to look for storage in: %s", len(content))
-    if len(content) < STORAGE_LOCATION + STORAGE_SIZE:
-        raise ConfigLengthError("provided content is not large enough to have a storage section!")
+    return get_storage_section(content, BOARD_CONFIG_BINARY_LOCATION)
 
-    logger.debug("returning bytes from %s to %s", hex(STORAGE_LOCATION), hex(STORAGE_LOCATION + STORAGE_SIZE))
-    return content[STORAGE_LOCATION:(STORAGE_LOCATION + STORAGE_SIZE)]
+
+def get_user_storage_section(content: bytes) -> bytes:
+    """Get the user storage area from what should be a whole board GP2040-CE dump.
+
+    Args:
+        content: bytes of a GP2040-CE whole board dump
+    Returns:
+        the presumed storage section from the binary
+    Raises:
+        ConfigLengthError: if the provided bytes don't appear to have a storage section
+    """
+    return get_storage_section(content, USER_CONFIG_BINARY_LOCATION)
+
+
+def get_new_config() -> Message:
+    """Wrap the creation of a new Config message.
+
+    Returns:
+        the initialized Config
+    """
+    config_pb2 = get_config_pb2()
+    return config_pb2.Config()
 
 
 def pad_config_to_storage_size(config: bytes) -> bytearray:
@@ -133,11 +369,48 @@ def pad_config_to_storage_size(config: bytes) -> bytearray:
 
     return bytearray(b'\x00' * bytes_to_pad) + bytearray(config)
 
+
+def serialize_config_with_footer(config: Message) -> bytearray:
+    """Given a config, generate the config footer as expected by GP2040-CE."""
+    config_bytes = config.SerializeToString()
+    config_size = bytes(reversed(config.ByteSize().to_bytes(4, 'big')))
+    config_crc = bytes(reversed(binascii.crc32(config_bytes).to_bytes(4, 'big')))
+    config_magic = FOOTER_MAGIC
+
+    return config_bytes + config_size + config_crc + config_magic
+
+
 ############
 # COMMANDS #
 ############
 
 
+def dump_config():
+    """Save the GP2040-CE's user configuration to a binary or UF2 file."""
+    parser = argparse.ArgumentParser(
+        description="Read the configuration section from a USB device and save it to a binary file.",
+        parents=[core_parser],
+    )
+    parser.add_argument('--board-config', action='store_true', default=False,
+                        help="dump the board config rather than the user config")
+    parser.add_argument('filename', help="file to save the GP2040-CE board's config section to --- if the "
+                                         "suffix is .uf2, it is saved in UF2 format, else it is a raw binary")
+    args, _ = parser.parse_known_args()
+    if args.board_config:
+        config, _, _ = get_board_config_from_usb()
+    else:
+        config, _, _ = get_user_config_from_usb()
+    binary_config = serialize_config_with_footer(config)
+    with open(args.filename, 'wb') as out_file:
+        if args.filename[-4:] == '.uf2':
+            # we must pad to storage start in order for the UF2 write addresses to make sense
+            out_file.write(convert_binary_to_uf2([
+                (USER_CONFIG_BINARY_LOCATION, pad_config_to_storage_size(binary_config)),
+            ]))
+        else:
+            out_file.write(binary_config)
+
+
 def visualize():
     """Print the contents of GP2040-CE's storage."""
     parser = argparse.ArgumentParser(
@@ -147,10 +420,24 @@ def visualize():
     )
     parser.add_argument('--whole-board', action='store_true', help="indicate the binary file is a whole board dump")
     parser.add_argument('--json', action='store_true', help="print the config out as a JSON document")
-    parser.add_argument('filename', help=".bin file of a GP2040-CE board's storage section, bytes 101FE000-10200000, "
-                                         "or of a GP2040-CE's whole board dump if --whole-board is specified")
+    parser.add_argument('--board-config', action='store_true', default=False,
+                        help="display the board config rather than the user config")
+    group = parser.add_mutually_exclusive_group(required=True)
+    group.add_argument('--usb', action='store_true', help="retrieve the config from a RP2040 board connected over USB "
+                                                          "and in BOOTSEL mode")
+    group.add_argument('--filename', help=".bin file of a GP2040-CE board's storage section, bytes "
+                                          "101F8000-10200000, or of a GP2040-CE's whole board dump "
+                                          "if --whole-board is specified")
     args, _ = parser.parse_known_args()
-    config = get_config_from_file(args.filename, whole_board=args.whole_board)
+
+    if args.usb:
+        if args.board_config:
+            config, _, _ = get_board_config_from_usb()
+        else:
+            config, _, _ = get_user_config_from_usb()
+    else:
+        config = get_config_from_file(args.filename, whole_board=args.whole_board, board_config=args.board_config)
+
     if args.json:
         print(MessageToJson(config))
     else:

pyproject.toml

@@ -1,30 +1,49 @@
 [build-system]
-requires = ["setuptools>=45", "setuptools_scm[toml]>=6.2"]
+requires = ["setuptools>=61.0", "setuptools_scm[toml]>=6.2"]
+build-backend = "setuptools.build_meta"
 
 [project]
 name = "gp2040ce-binary-tools"
-description = "Tools for working with GP2040-CE binary dumps."
+description = "Tools for working with GP2040-CE firmware and storage binaries."
 readme = "README.md"
-license = {text = "MIT"}
+license = {text = "GPL-3.0-or-later"}
 authors = [
     {name = "Brian S. Stephan", email = "bss@incorporeal.org"},
 ]
 requires-python = ">=3.9"
-dependencies = ["grpcio-tools"]
+dependencies = ["grpcio-tools", "pyusb", "textual"]
 dynamic = ["version"]
+classifiers = [
+    "Environment :: Console",
+    "Programming Language :: Python :: 3",
+    "License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)",
+    "Operating System :: OS Independent",
+    "Topic :: Utilities",
+]
+
+[project.urls]
+"Homepage" = "https://github.com/OpenStickCommunity/gp2040ce-binary-tools"
+"Changelog" = "https://github.com/OpenStickCommunity/gp2040ce-binary-tools/blob/main/CHANGELOG.md"
+"Bug Tracker" = "https://github.com/OpenStickCommunity/gp2040ce-binary-tools/issues"
+
 
 [project.optional-dependencies]
 dev = ["bandit", "decorator", "flake8", "flake8-blind-except", "flake8-builtins", "flake8-docstrings",
        "flake8-executable", "flake8-fixme", "flake8-isort", "flake8-logging-format", "flake8-mutable",
-       "flake8-pyproject", "mypy", "pip-tools", "pytest", "pytest-cov", "setuptools-scm", "tox"]
+       "flake8-pyproject", "mypy", "pip-tools", "pytest", "pytest-asyncio", "pytest-cov", "reuse",
+       "setuptools-scm", "textual-dev", "tox", "twine"]
 
 [project.scripts]
 concatenate = "gp2040ce_bintools.builder:concatenate"
-visualize-storage = "gp2040ce_bintools.storage:visualize"
+dump-config = "gp2040ce_bintools.storage:dump_config"
+dump-gp2040ce = "gp2040ce_bintools.builder:dump_gp2040ce"
+edit-config = "gp2040ce_bintools.gui:edit_config"
+summarize-gp2040ce = "gp2040ce_bintools.builder:summarize_gp2040ce"
+visualize-config = "gp2040ce_bintools.storage:visualize"
 
 [tool.flake8]
 enable-extensions = "G,M"
-exclude = [".tox/", "venv/", "_version.py", "tests/test-files/"]
+exclude = [".tox/", "venv/", "_version.py", "tests/test-files/", "config_pb2.py", "enums_pb2.py", "nanopb_pb2.py"]
 extend-ignore = "T101"
 max-complexity = 10
 max-line-length = 120
@@ -33,14 +52,37 @@ max-line-length = 120
 line_length = 120
 
 [tool.mypy]
+exclude = [
+    "config_pb2.py",
+    "enums_pb2.py",
+    "nanopb_pb2.py",
+]
 ignore_missing_imports = true
 
 [[tool.mypy.overrides]]
 module = "google.protobuf.*"
 ignore_missing_imports = true
 
+# there's a lot of class inheritance and so on going on in textual that I haven't figured out
+# e.g. assigning Select or Input to the same variable is valid but not type-safe, bindings
+# can just exit but mypy thinks they should return coroutines... better just to disable it for
+# now until I can figure things out and learn more about doing proper type checking
+[[tool.mypy.overrides]]
+module = "gp2040ce_bintools.gui"
+ignore_errors = true
+
 [tool.pytest]
 python_files = ["*_tests.py", "tests.py", "test_*.py"]
 
+[tool.pytest.ini_options]
+log_cli = 0
+log_cli_level = "WARNING"
+
+[tool.setuptools]
+packages = [
+    "gp2040ce_bintools",
+    "gp2040ce_bintools.proto_snapshot",
+]
+
 [tool.setuptools_scm]
 write_to = "gp2040ce_bintools/_version.py"

requirements/requirements-dev.txt

@@ -1,34 +1,69 @@
 #
-# This file is autogenerated by pip-compile with Python 3.10
+# This file is autogenerated by pip-compile with Python 3.12
 # by the following command:
 #
 #    pip-compile --extra=dev --output-file=requirements/requirements-dev.txt
 #
-bandit==1.7.5
+aiohappyeyeballs==2.4.4
+    # via aiohttp
+aiohttp==3.11.10
+    # via
+    #   aiohttp-jinja2
+    #   textual-dev
+    #   textual-serve
+aiohttp-jinja2==1.6
+    # via textual-serve
+aiosignal==1.3.2
+    # via aiohttp
+attrs==24.3.0
+    # via
+    #   aiohttp
+    #   reuse
+bandit==1.8.0
     # via gp2040ce-binary-tools (pyproject.toml)
-build==0.10.0
+binaryornot==0.4.4
+    # via reuse
+boolean-py==4.0
+    # via
+    #   license-expression
+    #   reuse
+build==1.2.2.post1
     # via pip-tools
-cachetools==5.3.1
+cachetools==5.5.0
     # via tox
-chardet==5.1.0
-    # via tox
-click==8.1.3
-    # via pip-tools
+certifi==2024.12.14
+    # via requests
+cffi==1.17.1
+    # via cryptography
+chardet==5.2.0
+    # via
+    #   binaryornot
+    #   python-debian
+    #   tox
+charset-normalizer==3.4.0
+    # via requests
+click==8.1.7
+    # via
+    #   pip-tools
+    #   reuse
+    #   textual-dev
 colorama==0.4.6
     # via tox
-coverage[toml]==7.2.7
+coverage[toml]==7.6.9
     # via pytest-cov
+cryptography==44.0.0
+    # via secretstorage
 decorator==5.1.1
     # via gp2040ce-binary-tools (pyproject.toml)
-distlib==0.3.6
+distlib==0.3.9
     # via virtualenv
-exceptiongroup==1.1.1
-    # via pytest
-filelock==3.12.2
+docutils==0.21.2
+    # via readme-renderer
+filelock==3.16.1
     # via
     #   tox
     #   virtualenv
-flake8==6.0.0
+flake8==7.1.1
     # via
     #   flake8-builtins
     #   flake8-docstrings
@@ -39,7 +74,7 @@ flake8==6.0.0
     #   gp2040ce-binary-tools (pyproject.toml)
 flake8-blind-except==0.2.1
     # via gp2040ce-binary-tools (pyproject.toml)
-flake8-builtins==2.1.0
+flake8-builtins==2.5.0
     # via gp2040ce-binary-tools (pyproject.toml)
 flake8-docstrings==1.7.0
     # via gp2040ce-binary-tools (pyproject.toml)
@@ -47,108 +82,200 @@ flake8-executable==2.1.3
     # via gp2040ce-binary-tools (pyproject.toml)
 flake8-fixme==1.1.1
     # via gp2040ce-binary-tools (pyproject.toml)
-flake8-isort==6.0.0
+flake8-isort==6.1.1
     # via gp2040ce-binary-tools (pyproject.toml)
-flake8-logging-format==0.9.0
+flake8-logging-format==2024.24.12
     # via gp2040ce-binary-tools (pyproject.toml)
 flake8-mutable==1.2.0
     # via gp2040ce-binary-tools (pyproject.toml)
 flake8-pyproject==1.2.3
     # via gp2040ce-binary-tools (pyproject.toml)
-gitdb==4.0.10
-    # via gitpython
-gitpython==3.1.31
-    # via bandit
-grpcio==1.54.2
+frozenlist==1.5.0
+    # via
+    #   aiohttp
+    #   aiosignal
+grpcio==1.68.1
     # via grpcio-tools
-grpcio-tools==1.54.2
+grpcio-tools==1.68.1
     # via gp2040ce-binary-tools (pyproject.toml)
+idna==3.10
+    # via
+    #   requests
+    #   yarl
 iniconfig==2.0.0
     # via pytest
-isort==5.12.0
+isort==5.13.2
     # via flake8-isort
-markdown-it-py==3.0.0
-    # via rich
+jaraco-classes==3.4.0
+    # via keyring
+jaraco-context==6.0.1
+    # via keyring
+jaraco-functools==4.1.0
+    # via keyring
+jeepney==0.8.0
+    # via
+    #   keyring
+    #   secretstorage
+jinja2==3.1.4
+    # via
+    #   aiohttp-jinja2
+    #   reuse
+    #   textual-serve
+keyring==25.5.0
+    # via twine
+license-expression==30.4.0
+    # via reuse
+linkify-it-py==2.0.3
+    # via markdown-it-py
+markdown-it-py[linkify,plugins]==3.0.0
+    # via
+    #   mdit-py-plugins
+    #   rich
+    #   textual
+markupsafe==3.0.2
+    # via jinja2
 mccabe==0.7.0
     # via flake8
+mdit-py-plugins==0.4.2
+    # via markdown-it-py
 mdurl==0.1.2
     # via markdown-it-py
-mypy==1.4.0
+more-itertools==10.5.0
+    # via
+    #   jaraco-classes
+    #   jaraco-functools
+msgpack==1.1.0
+    # via textual-dev
+multidict==6.1.0
+    # via
+    #   aiohttp
+    #   yarl
+mypy==1.13.0
     # via gp2040ce-binary-tools (pyproject.toml)
 mypy-extensions==1.0.0
     # via mypy
-packaging==23.1
+nh3==0.2.20
+    # via readme-renderer
+packaging==24.2
     # via
     #   build
     #   pyproject-api
     #   pytest
     #   setuptools-scm
     #   tox
-pbr==5.11.1
+    #   twine
+pbr==6.1.0
     # via stevedore
-pip-tools==6.13.0
+pip-tools==7.4.1
     # via gp2040ce-binary-tools (pyproject.toml)
-platformdirs==3.7.0
+pkginfo==1.12.0
+    # via twine
+platformdirs==4.3.6
     # via
+    #   textual
     #   tox
     #   virtualenv
-pluggy==1.2.0
+pluggy==1.5.0
     # via
     #   pytest
     #   tox
-protobuf==4.23.3
+propcache==0.2.1
+    # via
+    #   aiohttp
+    #   yarl
+protobuf==5.29.1
     # via grpcio-tools
-pycodestyle==2.10.0
+pycodestyle==2.12.1
     # via flake8
+pycparser==2.22
+    # via cffi
 pydocstyle==6.3.0
     # via flake8-docstrings
-pyflakes==3.0.1
+pyflakes==3.2.0
     # via flake8
-pygments==2.15.1
-    # via rich
-pyproject-api==1.5.2
-    # via tox
-pyproject-hooks==1.0.0
-    # via build
-pytest==7.3.2
+pygments==2.18.0
     # via
-    #   gp2040ce-binary-tools (pyproject.toml)
-    #   pytest-cov
-pytest-cov==4.1.0
-    # via gp2040ce-binary-tools (pyproject.toml)
-pyyaml==6.0
-    # via bandit
-rich==13.4.2
-    # via bandit
-setuptools-scm==7.1.0
-    # via gp2040ce-binary-tools (pyproject.toml)
-smmap==5.0.0
-    # via gitdb
-snowballstemmer==2.2.0
-    # via pydocstyle
-stevedore==5.1.0
-    # via bandit
-tomli==2.0.1
+    #   readme-renderer
+    #   rich
+pyproject-api==1.8.0
+    # via tox
+pyproject-hooks==1.2.0
     # via
     #   build
-    #   coverage
-    #   flake8-pyproject
-    #   mypy
-    #   pyproject-api
-    #   pyproject-hooks
-    #   pytest
-    #   setuptools-scm
-    #   tox
-tox==4.6.3
+    #   pip-tools
+pytest==8.3.4
+    # via
+    #   gp2040ce-binary-tools (pyproject.toml)
+    #   pytest-asyncio
+    #   pytest-cov
+pytest-asyncio==0.25.0
     # via gp2040ce-binary-tools (pyproject.toml)
-typing-extensions==4.6.3
+pytest-cov==6.0.0
+    # via gp2040ce-binary-tools (pyproject.toml)
+python-debian==0.1.49
+    # via reuse
+pyusb==1.2.1
+    # via gp2040ce-binary-tools (pyproject.toml)
+pyyaml==6.0.2
+    # via bandit
+readme-renderer==44.0
+    # via twine
+requests==2.32.3
+    # via
+    #   requests-toolbelt
+    #   twine
+requests-toolbelt==1.0.0
+    # via twine
+reuse==5.0.2
+    # via gp2040ce-binary-tools (pyproject.toml)
+rfc3986==2.0.0
+    # via twine
+rich==13.9.4
+    # via
+    #   bandit
+    #   textual
+    #   textual-serve
+    #   twine
+secretstorage==3.3.3
+    # via keyring
+setuptools-scm==8.1.0
+    # via gp2040ce-binary-tools (pyproject.toml)
+snowballstemmer==2.2.0
+    # via pydocstyle
+stevedore==5.4.0
+    # via bandit
+textual==1.0.0
+    # via
+    #   gp2040ce-binary-tools (pyproject.toml)
+    #   textual-dev
+    #   textual-serve
+textual-dev==1.7.0
+    # via gp2040ce-binary-tools (pyproject.toml)
+textual-serve==1.1.1
+    # via textual-dev
+tomlkit==0.13.2
+    # via reuse
+tox==4.23.2
+    # via gp2040ce-binary-tools (pyproject.toml)
+twine==6.0.1
+    # via gp2040ce-binary-tools (pyproject.toml)
+typing-extensions==4.12.2
     # via
     #   mypy
-    #   setuptools-scm
-virtualenv==20.23.1
+    #   textual
+    #   textual-dev
+uc-micro-py==1.0.3
+    # via linkify-it-py
+urllib3==2.2.3
+    # via
+    #   requests
+    #   twine
+virtualenv==20.28.0
     # via tox
-wheel==0.40.0
+wheel==0.45.1
     # via pip-tools
+yarl==1.18.3
+    # via aiohttp
 
 # The following packages are considered to be unsafe in a requirements file:
 # pip

requirements/requirements.txt

@@ -1,15 +1,40 @@
 #
-# This file is autogenerated by pip-compile with Python 3.10
+# This file is autogenerated by pip-compile with Python 3.12
 # by the following command:
 #
 #    pip-compile --output-file=requirements/requirements.txt
 #
-grpcio==1.54.2
+grpcio==1.68.1
     # via grpcio-tools
-grpcio-tools==1.54.2
+grpcio-tools==1.68.1
     # via gp2040ce-binary-tools (pyproject.toml)
-protobuf==4.23.3
+linkify-it-py==2.0.3
+    # via markdown-it-py
+markdown-it-py[linkify,plugins]==3.0.0
+    # via
+    #   mdit-py-plugins
+    #   rich
+    #   textual
+mdit-py-plugins==0.4.2
+    # via markdown-it-py
+mdurl==0.1.2
+    # via markdown-it-py
+platformdirs==4.3.6
+    # via textual
+protobuf==5.29.1
     # via grpcio-tools
+pygments==2.18.0
+    # via rich
+pyusb==1.2.1
+    # via gp2040ce-binary-tools (pyproject.toml)
+rich==13.9.4
+    # via textual
+textual==1.0.0
+    # via gp2040ce-binary-tools (pyproject.toml)
+typing-extensions==4.12.2
+    # via textual
+uc-micro-py==1.0.3
+    # via linkify-it-py
 
 # The following packages are considered to be unsafe in a requirements file:
 # setuptools

tests/conftest.py

@@ -1,4 +1,8 @@
-"""Create the test fixtures and other data."""
+"""Create the test fixtures and other data.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
 import os
 
 import pytest
@@ -16,6 +20,16 @@ def config_binary():
     yield content
 
 
+@pytest.fixture
+def config_json():
+    """Read in a test GP2040-CE configuration, Protobuf serialized binary form with footer."""
+    filename = os.path.join(HERE, 'test-files', 'test-config.json')
+    with open(filename, 'r') as file:
+        content = file.read()
+
+    yield content
+
+
 @pytest.fixture
 def firmware_binary():
     """Read in a test GP2040-CE firmware binary file."""
@@ -28,7 +42,7 @@ def firmware_binary():
 
 @pytest.fixture
 def storage_dump():
-    """Read in a test storage dump file (101FE000-10200000) of a GP2040-CE board."""
+    """Read in a test storage dump file (101F8000-10200000) of a GP2040-CE board."""
     filename = os.path.join(HERE, 'test-files', 'test-storage-area.bin')
     with open(filename, 'rb') as file:
         content = file.read()
@@ -38,9 +52,22 @@ def storage_dump():
 
 @pytest.fixture
 def whole_board_dump():
-    """Read in a test whole board dump file of a GP2040-CE board."""
+    """Read in a test whole board dump file of a GP2040-CE board.
+
+    NOTE: this is from a 16 MB flash because I used an ABB for this test.
+    """
     filename = os.path.join(HERE, 'test-files', 'test-whole-board.bin')
     with open(filename, 'rb') as file:
         content = file.read()
 
     yield content
+
+
+@pytest.fixture
+def whole_board_with_board_config_dump():
+    """Read in a test whole board dump file of a GP2040-CE board plus board config."""
+    filename = os.path.join(HERE, 'test-files', 'test-whole-board-with-board-config.bin')
+    with open(filename, 'rb') as file:
+        content = file.read()
+
+    yield content

tests/test-files/pb2-files/nanopb_pb2.py

@@ -1,11 +1,22 @@
 # -*- coding: utf-8 -*-
 # Generated by the protocol buffer compiler.  DO NOT EDIT!
+# NO CHECKED-IN PROTOBUF GENCODE
 # source: nanopb.proto
+# Protobuf Python Version: 5.27.2
 """Generated protocol buffer code."""
-from google.protobuf.internal import builder as _builder
 from google.protobuf import descriptor as _descriptor
 from google.protobuf import descriptor_pool as _descriptor_pool
+from google.protobuf import runtime_version as _runtime_version
 from google.protobuf import symbol_database as _symbol_database
+from google.protobuf.internal import builder as _builder
+_runtime_version.ValidateProtobufRuntimeVersion(
+    _runtime_version.Domain.PUBLIC,
+    5,
+    27,
+    2,
+    '',
+    'nanopb.proto'
+)
 # @@protoc_insertion_point(imports)
 
 _sym_db = _symbol_database.Default()
@@ -16,24 +27,20 @@ from google.protobuf import descriptor_pb2 as google_dot_protobuf_dot_descriptor
 
 DESCRIPTOR = _descriptor_pool.Default().AddSerializedFile(b'\n\x0cnanopb.proto\x1a google/protobuf/descriptor.proto\"\xc4\x07\n\rNanoPBOptions\x12\x10\n\x08max_size\x18\x01 \x01(\x05\x12\x12\n\nmax_length\x18\x0e \x01(\x05\x12\x11\n\tmax_count\x18\x02 \x01(\x05\x12&\n\x08int_size\x18\x07 \x01(\x0e\x32\x08.IntSize:\nIS_DEFAULT\x12$\n\x04type\x18\x03 \x01(\x0e\x32\n.FieldType:\nFT_DEFAULT\x12\x18\n\nlong_names\x18\x04 \x01(\x08:\x04true\x12\x1c\n\rpacked_struct\x18\x05 \x01(\x08:\x05\x66\x61lse\x12\x1a\n\x0bpacked_enum\x18\n \x01(\x08:\x05\x66\x61lse\x12\x1b\n\x0cskip_message\x18\x06 \x01(\x08:\x05\x66\x61lse\x12\x18\n\tno_unions\x18\x08 \x01(\x08:\x05\x66\x61lse\x12\r\n\x05msgid\x18\t \x01(\r\x12\x1e\n\x0f\x61nonymous_oneof\x18\x0b \x01(\x08:\x05\x66\x61lse\x12\x15\n\x06proto3\x18\x0c \x01(\x08:\x05\x66\x61lse\x12#\n\x14proto3_singular_msgs\x18\x15 \x01(\x08:\x05\x66\x61lse\x12\x1d\n\x0e\x65num_to_string\x18\r \x01(\x08:\x05\x66\x61lse\x12\x1b\n\x0c\x66ixed_length\x18\x0f \x01(\x08:\x05\x66\x61lse\x12\x1a\n\x0b\x66ixed_count\x18\x10 \x01(\x08:\x05\x66\x61lse\x12\x1e\n\x0fsubmsg_callback\x18\x16 \x01(\x08:\x05\x66\x61lse\x12/\n\x0cmangle_names\x18\x11 \x01(\x0e\x32\x11.TypenameMangling:\x06M_NONE\x12(\n\x11\x63\x61llback_datatype\x18\x12 \x01(\t:\rpb_callback_t\x12\x34\n\x11\x63\x61llback_function\x18\x13 \x01(\t:\x19pb_default_field_callback\x12\x30\n\x0e\x64\x65scriptorsize\x18\x14 \x01(\x0e\x32\x0f.DescriptorSize:\x07\x44S_AUTO\x12\x1a\n\x0b\x64\x65\x66\x61ult_has\x18\x17 \x01(\x08:\x05\x66\x61lse\x12\x0f\n\x07include\x18\x18 \x03(\t\x12\x0f\n\x07\x65xclude\x18\x1a \x03(\t\x12\x0f\n\x07package\x18\x19 \x01(\t\x12\x41\n\rtype_override\x18\x1b \x01(\x0e\x32*.google.protobuf.FieldDescriptorProto.Type\x12\x19\n\x0bsort_by_tag\x18\x1c \x01(\x08:\x04true\x12.\n\rfallback_type\x18\x1d \x01(\x0e\x32\n.FieldType:\x0b\x46T_CALLBACK\x12\x1e\n\x0f\x64isallow_export\x18\x1e \x01(\x08:\x05\x66\x61lse*i\n\tFieldType\x12\x0e\n\nFT_DEFAULT\x10\x00\x12\x0f\n\x0b\x46T_CALLBACK\x10\x01\x12\x0e\n\nFT_POINTER\x10\x04\x12\r\n\tFT_STATIC\x10\x02\x12\r\n\tFT_IGNORE\x10\x03\x12\r\n\tFT_INLINE\x10\x05*D\n\x07IntSize\x12\x0e\n\nIS_DEFAULT\x10\x00\x12\x08\n\x04IS_8\x10\x08\x12\t\n\x05IS_16\x10\x10\x12\t\n\x05IS_32\x10 \x12\t\n\x05IS_64\x10@*Z\n\x10TypenameMangling\x12\n\n\x06M_NONE\x10\x00\x12\x13\n\x0fM_STRIP_PACKAGE\x10\x01\x12\r\n\tM_FLATTEN\x10\x02\x12\x16\n\x12M_PACKAGE_INITIALS\x10\x03*E\n\x0e\x44\x65scriptorSize\x12\x0b\n\x07\x44S_AUTO\x10\x00\x12\x08\n\x04\x44S_1\x10\x01\x12\x08\n\x04\x44S_2\x10\x02\x12\x08\n\x04\x44S_4\x10\x04\x12\x08\n\x04\x44S_8\x10\x08:E\n\x0enanopb_fileopt\x12\x1c.google.protobuf.FileOptions\x18\xf2\x07 \x01(\x0b\x32\x0e.NanoPBOptions:G\n\rnanopb_msgopt\x12\x1f.google.protobuf.MessageOptions\x18\xf2\x07 \x01(\x0b\x32\x0e.NanoPBOptions:E\n\x0enanopb_enumopt\x12\x1c.google.protobuf.EnumOptions\x18\xf2\x07 \x01(\x0b\x32\x0e.NanoPBOptions:>\n\x06nanopb\x12\x1d.google.protobuf.FieldOptions\x18\xf2\x07 \x01(\x0b\x32\x0e.NanoPBOptionsB\x1a\n\x18\x66i.kapsi.koti.jpa.nanopb')
 
-_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, globals())
-_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'nanopb_pb2', globals())
-if _descriptor._USE_C_DESCRIPTORS == False:
-  google_dot_protobuf_dot_descriptor__pb2.FileOptions.RegisterExtension(nanopb_fileopt)
-  google_dot_protobuf_dot_descriptor__pb2.MessageOptions.RegisterExtension(nanopb_msgopt)
-  google_dot_protobuf_dot_descriptor__pb2.EnumOptions.RegisterExtension(nanopb_enumopt)
-  google_dot_protobuf_dot_descriptor__pb2.FieldOptions.RegisterExtension(nanopb)
-
-  DESCRIPTOR._options = None
-  DESCRIPTOR._serialized_options = b'\n\030fi.kapsi.koti.jpa.nanopb'
-  _FIELDTYPE._serialized_start=1017
-  _FIELDTYPE._serialized_end=1122
-  _INTSIZE._serialized_start=1124
-  _INTSIZE._serialized_end=1192
-  _TYPENAMEMANGLING._serialized_start=1194
-  _TYPENAMEMANGLING._serialized_end=1284
-  _DESCRIPTORSIZE._serialized_start=1286
-  _DESCRIPTORSIZE._serialized_end=1355
-  _NANOPBOPTIONS._serialized_start=51
-  _NANOPBOPTIONS._serialized_end=1015
+_globals = globals()
+_builder.BuildMessageAndEnumDescriptors(DESCRIPTOR, _globals)
+_builder.BuildTopDescriptorsAndMessages(DESCRIPTOR, 'nanopb_pb2', _globals)
+if not _descriptor._USE_C_DESCRIPTORS:
+  _globals['DESCRIPTOR']._loaded_options = None
+  _globals['DESCRIPTOR']._serialized_options = b'\n\030fi.kapsi.koti.jpa.nanopb'
+  _globals['_FIELDTYPE']._serialized_start=1017
+  _globals['_FIELDTYPE']._serialized_end=1122
+  _globals['_INTSIZE']._serialized_start=1124
+  _globals['_INTSIZE']._serialized_end=1192
+  _globals['_TYPENAMEMANGLING']._serialized_start=1194
+  _globals['_TYPENAMEMANGLING']._serialized_end=1284
+  _globals['_DESCRIPTORSIZE']._serialized_start=1286
+  _globals['_DESCRIPTORSIZE']._serialized_end=1355
+  _globals['_NANOPBOPTIONS']._serialized_start=51
+  _globals['_NANOPBOPTIONS']._serialized_end=1015
 # @@protoc_insertion_point(module_scope)

tests/test-files/proto-files/enums.proto

@@ -19,6 +19,27 @@ enum ButtonLayout
     BUTTON_LAYOUT_FIGHTBOARD_STICK = 10;
     BUTTON_LAYOUT_FIGHTBOARD_MIRRORED = 11;
     BUTTON_LAYOUT_CUSTOMA = 12;
+    BUTTON_LAYOUT_OPENCORE0WASDA = 13;
+    BUTTON_LAYOUT_STICKLESS_13 = 14;
+    BUTTON_LAYOUT_STICKLESS_16 = 15;
+    BUTTON_LAYOUT_STICKLESS_14 = 16;
+    BUTTON_LAYOUT_DANCEPAD_DDR_LEFT = 17;
+    BUTTON_LAYOUT_DANCEPAD_DDR_SOLO = 18;
+    BUTTON_LAYOUT_DANCEPAD_PIU_LEFT = 19;
+    BUTTON_LAYOUT_POPN_A = 20;
+    BUTTON_LAYOUT_TAIKO_A = 21;
+    BUTTON_LAYOUT_BM_TURNTABLE_A = 22;
+    BUTTON_LAYOUT_BM_5KEY_A = 23;
+    BUTTON_LAYOUT_BM_7KEY_A = 24;
+    BUTTON_LAYOUT_GITADORA_FRET_A = 25;
+    BUTTON_LAYOUT_GITADORA_STRUM_A = 26;
+    BUTTON_LAYOUT_BOARD_DEFINED_A = 27;
+    BUTTON_LAYOUT_BANDHERO_FRET_A = 28;
+    BUTTON_LAYOUT_BANDHERO_STRUM_A = 29;
+    BUTTON_LAYOUT_6GAWD_A = 30;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTON_A = 31;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTONPLUS_A = 32;
+    BUTTON_LAYOUT_STICKLESS_R16 = 33;
 }
 
 enum ButtonLayoutRight
@@ -42,6 +63,27 @@ enum ButtonLayoutRight
     BUTTON_LAYOUT_FIGHTBOARD = 14;
     BUTTON_LAYOUT_FIGHTBOARD_STICK_MIRRORED = 15;
     BUTTON_LAYOUT_CUSTOMB = 16;
+    BUTTON_LAYOUT_KEYBOARD8B = 17;
+    BUTTON_LAYOUT_OPENCORE0WASDB = 18;
+    BUTTON_LAYOUT_STICKLESS_13B = 19;
+    BUTTON_LAYOUT_STICKLESS_16B = 20;
+    BUTTON_LAYOUT_STICKLESS_14B = 21;
+    BUTTON_LAYOUT_DANCEPAD_DDR_RIGHT = 22;
+    BUTTON_LAYOUT_DANCEPAD_PIU_RIGHT = 23;
+    BUTTON_LAYOUT_POPN_B = 24;
+    BUTTON_LAYOUT_TAIKO_B = 25;
+    BUTTON_LAYOUT_BM_TURNTABLE_B = 26;
+    BUTTON_LAYOUT_BM_5KEY_B = 27;
+    BUTTON_LAYOUT_BM_7KEY_B = 28;
+    BUTTON_LAYOUT_GITADORA_FRET_B = 29;
+    BUTTON_LAYOUT_GITADORA_STRUM_B = 30;
+    BUTTON_LAYOUT_BOARD_DEFINED_B = 31;
+    BUTTON_LAYOUT_BANDHERO_FRET_B = 32;
+    BUTTON_LAYOUT_BANDHERO_STRUM_B = 33;
+    BUTTON_LAYOUT_6GAWD_B = 34;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTON_B = 35;
+    BUTTON_LAYOUT_6GAWD_ALLBUTTONPLUS_B = 36;
+    BUTTON_LAYOUT_STICKLESS_R16B = 37;
 }
 
 enum SplashMode
@@ -69,10 +111,11 @@ enum SplashChoice
 enum OnBoardLedMode
 {
     option (nanopb_enumopt).long_names = false;
-    
+
     ON_BOARD_LED_MODE_OFF = 0;
     ON_BOARD_LED_MODE_MODE_INDICATOR = 1;
     ON_BOARD_LED_MODE_INPUT_TEST = 2;
+    ON_BOARD_LED_MODE_PS_AUTH = 3;
 }
 
 enum InputMode
@@ -81,12 +124,32 @@ enum InputMode
 
     INPUT_MODE_XINPUT = 0;
     INPUT_MODE_SWITCH = 1;
-    INPUT_MODE_HID = 2;
+    INPUT_MODE_PS3 = 2;
     INPUT_MODE_KEYBOARD = 3;
     INPUT_MODE_PS4 = 4;
+    INPUT_MODE_XBONE = 5;
+    INPUT_MODE_MDMINI = 6;
+    INPUT_MODE_NEOGEO = 7;
+    INPUT_MODE_PCEMINI = 8;
+    INPUT_MODE_EGRET = 9;
+    INPUT_MODE_ASTRO = 10;
+    INPUT_MODE_PSCLASSIC = 11;
+    INPUT_MODE_XBOXORIGINAL = 12;
+    INPUT_MODE_PS5 = 13;
+    INPUT_MODE_GENERIC = 14;
     INPUT_MODE_CONFIG = 255;
 }
 
+enum InputModeAuthType
+{
+    option (nanopb_enumopt).long_names = false;
+
+    INPUT_MODE_AUTH_TYPE_NONE = 0;
+    INPUT_MODE_AUTH_TYPE_KEYS = 1;
+    INPUT_MODE_AUTH_TYPE_USB = 2;
+    INPUT_MODE_AUTH_TYPE_I2C = 3;
+}
+
 enum DpadMode
 {
     option (nanopb_enumopt).long_names = false;
@@ -96,6 +159,16 @@ enum DpadMode
     DPAD_MODE_RIGHT_ANALOG = 2;
 }
 
+enum InvertMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    INVERT_NONE = 0;
+    INVERT_X = 1;
+    INVERT_Y = 2;
+    INVERT_XY = 3;
+}
+
 enum SOCDMode
 {
     option (nanopb_enumopt).long_names = false;
@@ -107,6 +180,79 @@ enum SOCDMode
     SOCD_MODE_BYPASS = 4;					// U+D=UD, L+R=LR (No cleaning applied)
 }
 
+enum GpioAction
+{
+    option (nanopb_enumopt).long_names = false;
+
+    // the lowest value is the default, which should be NONE;
+    // reserving some numbers in case we need more not-mapped type values
+    NONE                         = -10;
+    RESERVED                     = -5;
+    ASSIGNED_TO_ADDON            = 0;
+    BUTTON_PRESS_UP              = 1;
+    BUTTON_PRESS_DOWN            = 2;
+    BUTTON_PRESS_LEFT            = 3;
+    BUTTON_PRESS_RIGHT           = 4;
+    BUTTON_PRESS_B1              = 5;
+    BUTTON_PRESS_B2              = 6;
+    BUTTON_PRESS_B3              = 7;
+    BUTTON_PRESS_B4              = 8;
+    BUTTON_PRESS_L1              = 9;
+    BUTTON_PRESS_R1              = 10;
+    BUTTON_PRESS_L2              = 11;
+    BUTTON_PRESS_R2              = 12;
+    BUTTON_PRESS_S1              = 13;
+    BUTTON_PRESS_S2              = 14;
+    BUTTON_PRESS_A1              = 15;
+    BUTTON_PRESS_A2              = 16;
+    BUTTON_PRESS_L3              = 17;
+    BUTTON_PRESS_R3              = 18;
+    BUTTON_PRESS_FN              = 19;
+    BUTTON_PRESS_DDI_UP          = 20;
+    BUTTON_PRESS_DDI_DOWN        = 21;
+    BUTTON_PRESS_DDI_LEFT        = 22;
+    BUTTON_PRESS_DDI_RIGHT       = 23;
+    SUSTAIN_DP_MODE_DP           = 24;
+    SUSTAIN_DP_MODE_LS           = 25;
+    SUSTAIN_DP_MODE_RS           = 26;
+    SUSTAIN_SOCD_MODE_UP_PRIO    = 27;
+    SUSTAIN_SOCD_MODE_NEUTRAL    = 28;
+    SUSTAIN_SOCD_MODE_SECOND_WIN = 29;
+    SUSTAIN_SOCD_MODE_FIRST_WIN  = 30;
+    SUSTAIN_SOCD_MODE_BYPASS     = 31;
+    BUTTON_PRESS_TURBO           = 32;
+    BUTTON_PRESS_MACRO           = 33;
+    BUTTON_PRESS_MACRO_1         = 34;
+    BUTTON_PRESS_MACRO_2         = 35;
+    BUTTON_PRESS_MACRO_3         = 36;
+    BUTTON_PRESS_MACRO_4         = 37;
+    BUTTON_PRESS_MACRO_5         = 38;
+    BUTTON_PRESS_MACRO_6         = 39;
+    CUSTOM_BUTTON_COMBO          = 40;
+    BUTTON_PRESS_A3              = 41;
+    BUTTON_PRESS_A4              = 42;
+    BUTTON_PRESS_E1              = 43;
+    BUTTON_PRESS_E2              = 44;
+    BUTTON_PRESS_E3              = 45;
+    BUTTON_PRESS_E4              = 46;
+    BUTTON_PRESS_E5              = 47;
+    BUTTON_PRESS_E6              = 48;
+    BUTTON_PRESS_E7              = 49;
+    BUTTON_PRESS_E8              = 50;
+    BUTTON_PRESS_E9              = 51;
+    BUTTON_PRESS_E10             = 52;
+    BUTTON_PRESS_E11             = 53;
+    BUTTON_PRESS_E12             = 54;
+}
+
+enum GpioDirection
+{
+    option (nanopb_enumopt).long_names = false;
+
+    GPIO_DIRECTION_INPUT = 0;
+    GPIO_DIRECTION_OUTPUT = 1;
+}
+
 enum GamepadHotkey
 {
     option (nanopb_enumopt).long_names = false;
@@ -126,6 +272,34 @@ enum GamepadHotkey
     HOTKEY_SOCD_BYPASS           = 12;
     HOTKEY_TOGGLE_4_WAY_MODE     = 13;
     HOTKEY_TOGGLE_DDI_4_WAY_MODE = 14;
+    HOTKEY_LOAD_PROFILE_1        = 15;
+    HOTKEY_LOAD_PROFILE_2        = 16;
+    HOTKEY_LOAD_PROFILE_3        = 17;
+    HOTKEY_LOAD_PROFILE_4        = 18;
+    HOTKEY_L3_BUTTON             = 19;
+    HOTKEY_R3_BUTTON             = 20;
+    HOTKEY_TOUCHPAD_BUTTON       = 21;
+    HOTKEY_REBOOT_DEFAULT        = 22;
+    HOTKEY_B1_BUTTON             = 23;
+    HOTKEY_B2_BUTTON             = 24;
+    HOTKEY_B3_BUTTON             = 25;
+    HOTKEY_B4_BUTTON             = 26;
+    HOTKEY_L1_BUTTON             = 27;
+    HOTKEY_R1_BUTTON             = 28;
+    HOTKEY_L2_BUTTON             = 29;
+    HOTKEY_R2_BUTTON             = 30;
+    HOTKEY_S1_BUTTON             = 31;
+    HOTKEY_S2_BUTTON             = 32;
+    HOTKEY_A1_BUTTON             = 33;
+    HOTKEY_A2_BUTTON             = 34;
+    HOTKEY_NEXT_PROFILE          = 35;
+    HOTKEY_A3_BUTTON             = 36;
+    HOTKEY_A4_BUTTON             = 37;
+    HOTKEY_DPAD_UP               = 38;
+    HOTKEY_DPAD_DOWN             = 39;
+    HOTKEY_DPAD_LEFT             = 40;
+    HOTKEY_DPAD_RIGHT            = 41;
+    HOTKEY_PREVIOUS_PROFILE      = 42;
 }
 
 // This has to be kept in sync with LEDFormat in NeoPico.hpp
@@ -148,7 +322,7 @@ enum ShmupMixMode
 enum PLEDType
 {
     option (nanopb_enumopt).long_names = false;
-    
+
     PLED_TYPE_NONE = -1;
     PLED_TYPE_PWM = 0;
     PLED_TYPE_RGB = 1;
@@ -157,9 +331,95 @@ enum PLEDType
 enum ForcedSetupMode
 {
     option (nanopb_enumopt).long_names = false;
-    
+
 	FORCED_SETUP_MODE_OFF = 0;
 	FORCED_SETUP_MODE_LOCK_MODE_SWITCH = 1;
 	FORCED_SETUP_MODE_LOCK_WEB_CONFIG = 2;
 	FORCED_SETUP_MODE_LOCK_BOTH = 3;
 };
+
+enum DualDirectionalCombinationMode
+{
+	option (nanopb_enumopt).long_names = false;
+
+	MIXED_MODE = 0;
+	GAMEPAD_MODE = 1;
+	DUAL_MODE = 2;
+	NONE_MODE = 3;
+}
+
+enum PS4ControllerType
+{
+    option (nanopb_enumopt).long_names = false;
+
+    PS4_CONTROLLER = 0;
+    PS4_ARCADESTICK = 7;
+}
+
+enum MacroType
+{
+    option (nanopb_enumopt).long_names = false;
+
+    ON_PRESS = 1;
+    ON_HOLD_REPEAT = 2;
+    ON_TOGGLE = 3;
+};
+
+enum GPElement
+{
+    option (nanopb_enumopt).long_names = false;
+
+    GP_ELEMENT_WIDGET = 0;
+    GP_ELEMENT_SCREEN = 1;
+    GP_ELEMENT_BTN_BUTTON = 2;
+    GP_ELEMENT_DIR_BUTTON = 3;
+    GP_ELEMENT_PIN_BUTTON = 4;
+    GP_ELEMENT_LEVER = 5;
+    GP_ELEMENT_LABEL = 6;
+    GP_ELEMENT_SPRITE = 7;
+    GP_ELEMENT_SHAPE = 8;
+};
+
+enum GPShape_Type
+{
+    option (nanopb_enumopt).long_names = false;
+
+    GP_SHAPE_ELLIPSE = 0;
+    GP_SHAPE_SQUARE = 1;
+    GP_SHAPE_LINE = 2;
+    GP_SHAPE_POLYGON = 3;
+    GP_SHAPE_ARC = 4;
+};
+
+enum RotaryEncoderPinMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    ENCODER_MODE_NONE = 0;
+    ENCODER_MODE_LEFT_ANALOG_X = 1;
+    ENCODER_MODE_LEFT_ANALOG_Y = 2;
+    ENCODER_MODE_RIGHT_ANALOG_X = 3;
+    ENCODER_MODE_RIGHT_ANALOG_Y = 4;
+    ENCODER_MODE_LEFT_TRIGGER = 5;
+    ENCODER_MODE_RIGHT_TRIGGER = 6;
+    ENCODER_MODE_DPAD_X = 7;
+    ENCODER_MODE_DPAD_Y = 8;
+};
+
+enum ReactiveLEDMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    REACTIVE_LED_STATIC_OFF = 0;
+    REACTIVE_LED_STATIC_ON = 1;
+    REACTIVE_LED_FADE_IN = 2;
+    REACTIVE_LED_FADE_OUT = 3;
+};
+
+enum PS4ControllerIDMode
+{
+    option (nanopb_enumopt).long_names = false;
+
+    PS4_ID_CONSOLE = 0;
+    PS4_ID_EMULATION = 1;
+};

tests/test_builder.py

@@ -1,35 +1,403 @@
-"""Tests for the image builder module."""
-import pytest
+"""Tests for the image builder module.
 
-from gp2040ce_bintools.builder import FirmwareLengthError, combine_firmware_and_config, pad_firmware_up_to_storage
-from gp2040ce_bintools.storage import get_config_footer, get_storage_section
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
+import logging
+import math
+import os
+import sys
+import unittest.mock as mock
+
+import pytest
+from decorator import decorator
+
+import gp2040ce_bintools.builder as builder
+from gp2040ce_bintools import get_config_pb2
+from gp2040ce_bintools.storage import (STORAGE_SIZE, get_board_storage_section, get_config, get_config_footer,
+                                       get_config_from_json, get_user_storage_section, serialize_config_with_footer)
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+logger = logging.getLogger(__name__)
+
+
+@decorator
+def with_pb2s(test, *args, **kwargs):
+    """Wrap a test with precompiled pb2 files on the path."""
+    proto_path = os.path.join(HERE, 'test-files', 'pb2-files')
+    sys.path.append(proto_path)
+
+    test(*args, **kwargs)
+
+    sys.path.pop()
+    del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
+
+
+def test_concatenate_to_file(tmp_path):
+    """Test that we write a file with firmware + binary user config as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.bin')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    assert len(content) == 2 * 1024 * 1024
+
+
+def test_concatenate_board_config_to_file(tmp_path):
+    """Test that we write a file with firmware + binary board config as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.bin')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    assert len(content) == (2 * 1024 * 1024) - (32 * 1024)
+
+
+def test_concatenate_both_configs_to_file(tmp_path):
+    """Test that we write a file with firmware + binary board + binary user config as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.bin')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   binary_user_config_filename=config_file, combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    assert len(content) == 2 * 1024 * 1024
+    storage = get_board_storage_section(content)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
+    storage = get_user_storage_section(content)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
+
+
+@with_pb2s
+def test_concatenate_user_json_to_file(tmp_path):
+    """Test that we write a file with firmware + JSON user config as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.bin')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.json')
+    builder.concatenate_firmware_and_storage_files(firmware_file, json_user_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    assert len(content) == 2 * 1024 * 1024
+
+
+def test_concatenate_to_file_incomplete_args_is_error(tmp_path):
+    """Test that we bail properly if we weren't given all the necessary arguments to make a binary."""
+    tmp_file = os.path.join(tmp_path, 'concat.bin')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    with pytest.raises(ValueError):
+        builder.concatenate_firmware_and_storage_files(firmware_file, combined_filename=tmp_file)
+
+
+def test_concatenate_to_usb(tmp_path):
+    """Test that we write a file as expected."""
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    with mock.patch('gp2040ce_bintools.builder.get_bootsel_endpoints', return_value=(end_out, end_in)):
+        with mock.patch('gp2040ce_bintools.builder.write') as mock_write:
+            builder.concatenate_firmware_and_storage_files(firmware_file, binary_user_config_filename=config_file,
+                                                           usb=True)
+
+    assert mock_write.call_args.args[2] == 0x10000000
+    assert len(mock_write.call_args.args[3]) == 2 * 1024 * 1024
+
+
+def test_concatenate_to_uf2(tmp_path, firmware_binary, config_binary):
+    """Test that we write a UF2 file as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.uf2')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   binary_user_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    # size of the file should be 2x the padded firmware + 2x the board config space + 2x the user config space
+    assert len(content) == (math.ceil(len(firmware_binary)/256) * 512 +
+                            math.ceil(STORAGE_SIZE/256) * 512 * 2)
+
+
+def test_concatenate_to_uf2_board_only(tmp_path, firmware_binary, config_binary):
+    """Test that we write a UF2 file as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.uf2')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    # size of the file should be 2x the padded firmware + 2x the board config space
+    assert len(content) == (math.ceil(len(firmware_binary)/256) * 512 +
+                            math.ceil(STORAGE_SIZE/256) * 512)
+
+
+def test_concatenate_with_backup(tmp_path, firmware_binary, config_binary):
+    """Test that we write a UF2 file as expected."""
+    tmp_file = os.path.join(tmp_path, 'concat.uf2')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    # create the file we are going to try to overwrite and want backed up
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   combined_filename=tmp_file)
+    # second file, expecting an overwrite of the target with a backup made
+    builder.concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                                   binary_user_config_filename=config_file,
+                                                   combined_filename=tmp_file,
+                                                   backup=True)
+    # size of the file should be 2x the padded firmware + 2x the board config space + 2x the user config space
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    assert len(content) == (math.ceil(len(firmware_binary)/256) * 512 +
+                            math.ceil(STORAGE_SIZE/256) * 512 * 2)
+    # size of the backup file should be 2x the padded firmware + 2x the board config space
+    with open(f'{tmp_file}.old', 'rb') as file:
+        content = file.read()
+    assert len(content) == (math.ceil(len(firmware_binary)/256) * 512 +
+                            math.ceil(STORAGE_SIZE/256) * 512)
+
+
+def test_find_version_string(firmware_binary):
+    """Test that we can find a version string in a binary."""
+    assert builder.find_version_string_in_binary(firmware_binary) == 'v0.7.5'
+
+
+def test_dont_always_find_version_string(firmware_binary):
+    """Test that we can find a version string in a binary."""
+    assert builder.find_version_string_in_binary(b'\x00') == 'NONE'
 
 
 def test_padding_firmware(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    padded = pad_firmware_up_to_storage(firmware_binary)
-    assert len(padded) == 2088960
+    padded = builder.pad_binary_up_to_user_config(firmware_binary)
+    assert len(padded) == 2064384
 
 
-def test_firmware_plus_storage(firmware_binary, storage_dump):
+def test_padding_firmware_can_truncate():
+    """Test that firmware is padded to the expected size."""
+    padded = builder.pad_binary_up_to_user_config(bytearray(b'\x00' * 4 * 1024 * 1024), or_truncate=True)
+    assert len(padded) == 2064384
+
+
+def test_padding_firmware_to_board(firmware_binary):
+    """Test that firmware is padded to the expected size."""
+    padded = builder.pad_binary_up_to_board_config(firmware_binary)
+    assert len(padded) == 2064384 - (32 * 1024)
+
+
+def test_firmware_plus_storage_section(firmware_binary, storage_dump):
     """Test that combining firmware and storage produces a valid combined binary."""
-    whole_board = combine_firmware_and_config(firmware_binary, storage_dump)
+    whole_board = builder.combine_firmware_and_config(firmware_binary, None, storage_dump)
     # if this is valid, we should be able to find the storage and footer again
-    storage = get_storage_section(whole_board)
+    storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
-    assert footer_size == 2032
+    assert footer_size == 3309
 
 
-def test_firmware_plus_config_binary(firmware_binary, config_binary):
-    """Test that combining firmware and storage produces a valid combined binary."""
-    whole_board = combine_firmware_and_config(firmware_binary, config_binary)
+def test_firmware_plus_user_config_binary(firmware_binary, config_binary):
+    """Test that combining firmware and user config produces a valid combined binary."""
+    whole_board = builder.combine_firmware_and_config(firmware_binary, None, config_binary)
     # if this is valid, we should be able to find the storage and footer again
-    storage = get_storage_section(whole_board)
+    storage = get_user_storage_section(whole_board)
     footer_size, _, _ = get_config_footer(storage)
-    assert footer_size == 2032
+    assert footer_size == 3309
+
+
+def test_chunky_firmware_plus_user_config_binary(config_binary):
+    """Test that combining giant firmware and storage produces a valid combined binary."""
+    whole_board = builder.combine_firmware_and_config(bytearray(b'\x00' * 4 * 1024 * 1024), None, config_binary,
+                                                      replace_extra=True)
+    # if this is valid, we should be able to find the storage and footer again
+    storage = get_user_storage_section(whole_board)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
+
+
+def test_firmware_plus_board_config_binary(firmware_binary, config_binary):
+    """Test that combining firmware and board config produces a valid combined binary."""
+    almost_whole_board = builder.combine_firmware_and_config(firmware_binary, config_binary, None)
+    assert len(almost_whole_board) == (2 * 1024 * 1024) - (32 * 1024)
+    # if this is valid, we should be able to find the storage and footer again
+    storage = get_board_storage_section(almost_whole_board)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
+
+
+def test_firmware_plus_board_and_user_config_binary(firmware_binary, config_binary):
+    """Test that combining firmware and both board and user configs produces a valid combined binary."""
+    whole_board = builder.combine_firmware_and_config(firmware_binary, config_binary, config_binary)
+    assert len(whole_board) == 2 * 1024 * 1024
+    # if this is valid, we should be able to find the storage and footer again
+    storage = get_board_storage_section(whole_board)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
+    storage = get_user_storage_section(whole_board)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
+
+
+def test_combine_must_get_at_least_one_config(firmware_binary):
+    """Test that we error if we are asked to combine with nothing to combine."""
+    with pytest.raises(ValueError):
+        builder.combine_firmware_and_config(firmware_binary, None, None)
+
+
+def test_replace_config_in_binary(config_binary):
+    """Test that a config binary is placed in the storage location of a source binary to overwrite."""
+    whole_board = builder.replace_config_in_binary(bytearray(b'\x00' * 3 * 1024 * 1024), config_binary)
+    assert len(whole_board) == 3 * 1024 * 1024
+    # if this is valid, we should be able to find the storage and footer again
+    storage = get_user_storage_section(whole_board)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
+
+
+def test_replace_config_in_binary_not_big_enough(config_binary):
+    """Test that a config binary is placed in the storage location of a source binary to pad."""
+    whole_board = builder.replace_config_in_binary(bytearray(b'\x00' * 1 * 1024 * 1024), config_binary)
+    assert len(whole_board) == 2 * 1024 * 1024
+    # if this is valid, we should be able to find the storage and footer again
+    storage = get_user_storage_section(whole_board)
+    footer_size, _, _ = get_config_footer(storage)
+    assert footer_size == 3309
 
 
 def test_padding_firmware_too_big(firmware_binary):
     """Test that firmware is padded to the expected size."""
-    with pytest.raises(FirmwareLengthError):
-        _ = pad_firmware_up_to_storage(firmware_binary + firmware_binary + firmware_binary)
+    with pytest.raises(builder.FirmwareLengthError):
+        _ = builder.pad_binary_up_to_user_config(firmware_binary + firmware_binary + firmware_binary)
+
+
+@with_pb2s
+def test_write_new_config_to_whole_board(whole_board_dump, tmp_path):
+    """Test that the config can be overwritten on a whole board dump."""
+    tmp_file = os.path.join(tmp_path, 'whole-board-dump-copy.bin')
+    with open(tmp_file, 'wb') as file:
+        file.write(whole_board_dump)
+    # reread just in case
+    with open(tmp_file, 'rb') as file:
+        board_dump = file.read()
+
+    config = get_config(get_user_storage_section(board_dump))
+    assert config.boardVersion == 'v0.7.5'
+    config.boardVersion = 'v0.7.5-COOL'
+    builder.write_new_config_to_filename(config, tmp_file, inject=True)
+
+    # read new file
+    with open(tmp_file, 'rb') as file:
+        new_board_dump = file.read()
+    config = get_config(get_user_storage_section(new_board_dump))
+    assert config.boardVersion == 'v0.7.5-COOL'
+    assert len(board_dump) == len(new_board_dump)
+
+
+@with_pb2s
+def test_write_new_config_to_firmware(firmware_binary, tmp_path):
+    """Test that the config can be added on a firmware."""
+    tmp_file = os.path.join(tmp_path, 'firmware-copy.bin')
+    with open(tmp_file, 'wb') as file:
+        file.write(firmware_binary)
+
+    config_pb2 = get_config_pb2()
+    config = config_pb2.Config()
+    config.boardVersion = 'v0.7.5-COOL'
+    builder.write_new_config_to_filename(config, tmp_file, inject=True)
+
+    # read new file
+    with open(tmp_file, 'rb') as file:
+        new_board_dump = file.read()
+    config = get_config(get_user_storage_section(new_board_dump))
+    assert config.boardVersion == 'v0.7.5-COOL'
+    assert len(new_board_dump) == 2 * 1024 * 1024
+
+
+@with_pb2s
+def test_write_new_config_to_config_bin(firmware_binary, tmp_path):
+    """Test that the config can be written to a file."""
+    tmp_file = os.path.join(tmp_path, 'config.bin')
+    config_pb2 = get_config_pb2()
+    config = config_pb2.Config()
+    config.boardVersion = 'v0.7.5-COOL'
+    builder.write_new_config_to_filename(config, tmp_file)
+
+    # read new file
+    with open(tmp_file, 'rb') as file:
+        config_dump = file.read()
+    config = get_config(config_dump)
+    config_size, _, _ = get_config_footer(config_dump)
+    assert config.boardVersion == 'v0.7.5-COOL'
+    assert len(config_dump) == config_size + 12
+
+
+@with_pb2s
+def test_write_new_config_to_config_uf2(firmware_binary, tmp_path):
+    """Test that the config can be written to a file."""
+    tmp_file = os.path.join(tmp_path, 'config.uf2')
+    config_pb2 = get_config_pb2()
+    config = config_pb2.Config()
+    config.boardVersion = 'v0.7.5-COOL'
+    builder.write_new_config_to_filename(config, tmp_file)
+
+    # read new file
+    with open(tmp_file, 'rb') as file:
+        config_dump = file.read()
+    # the current implementation of UF2 writing does it in 256 blocks, so each 256 byte block of
+    # binary is 512 bytes in the UF2
+    assert len(config_dump) == STORAGE_SIZE * 2
+
+
+@with_pb2s
+def test_write_new_config_to_config_json(config_binary, tmp_path):
+    """Test that the config can be written to a file."""
+    tmp_file = os.path.join(tmp_path, 'config.json')
+    config = get_config(config_binary)
+    builder.write_new_config_to_filename(config, tmp_file)
+
+    # read new file
+    with open(tmp_file, 'r') as file:
+        config_dump = file.read()
+    logger.debug(config_dump)
+    config = get_config_from_json(config_dump)
+    assert config.boardVersion == 'v0.7.5'
+
+
+@with_pb2s
+def test_write_new_config_to_usb(config_binary):
+    """Test that the config can be written to USB at the proper alignment."""
+    config = get_config(config_binary)
+    serialized = serialize_config_with_footer(config)
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    with mock.patch('gp2040ce_bintools.builder.write') as mock_write:
+        builder.write_new_config_to_usb(config, end_out, end_in)
+
+    # check that it got padded
+    assert len(serialized) == 3321
+    padded_serialized = bytearray(b'\x00' * 775) + serialized
+    assert mock_write.call_args.args[2] % 4096 == 0
+    assert mock_write.call_args.args[3] == padded_serialized
+
+
+def test_get_gp2040ce_from_usb():
+    """Test we attempt to read from the proper location over USB."""
+    mock_out = mock.MagicMock()
+    mock_out.device.idVendor = 0xbeef
+    mock_out.device.idProduct = 0xcafe
+    mock_out.device.bus = 1
+    mock_out.device.address = 2
+    mock_in = mock.MagicMock()
+    with mock.patch('gp2040ce_bintools.builder.get_bootsel_endpoints', return_value=(mock_out, mock_in)) as mock_get:
+        with mock.patch('gp2040ce_bintools.builder.read') as mock_read:
+            config, _, _ = builder.get_gp2040ce_from_usb()
+
+    mock_get.assert_called_once()
+    mock_read.assert_called_with(mock_out, mock_in, 0x10000000, 2 * 1024 * 1024)

tests/test_commands.py

@@ -1,55 +1,100 @@
-"""Test our tools themselves to make sure they adhere to certain flags."""
+"""Test our tools themselves to make sure they adhere to certain flags.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
 import json
 import os
+import sys
 from subprocess import run
 
+from decorator import decorator
+
 from gp2040ce_bintools import __version__
 
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+
+@decorator
+def with_pb2s(test, *args, **kwargs):
+    """Wrap a test with precompiled pb2 files on the path."""
+    proto_path = os.path.join(HERE, 'test-files', 'pb2-files')
+    sys.path.append(proto_path)
+
+    test(*args, **kwargs)
+
+    sys.path.pop()
+    del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
+
 
 def test_version_flag():
     """Test that tools report the version."""
-    result = run(['visualize-storage', '-v'], capture_output=True, encoding='utf8')
-    assert __version__ in result.stdout
+    result = run(['visualize-config', '-v'], capture_output=True, encoding='utf8')
+    assert f'gp2040ce-binary-tools {__version__}' in result.stdout
+    assert 'Python 3' in result.stdout
 
 
 def test_help_flag():
     """Test that tools report the usage information."""
-    result = run(['visualize-storage', '-h'], capture_output=True, encoding='utf8')
-    assert 'usage: visualize-storage' in result.stdout
+    result = run(['visualize-config', '-h'], capture_output=True, encoding='utf8')
+    assert 'usage: visualize-config' in result.stdout
     assert 'Read the configuration section from a dump of a GP2040-CE board' in result.stdout
 
 
 def test_concatenate_invocation(tmpdir):
     """Test that a normal invocation against a dump works."""
     out_filename = os.path.join(tmpdir, 'out.bin')
-    _ = run(['concatenate', 'tests/test-files/test-firmware.bin', 'tests/test-files/test-storage-area.bin',
-             out_filename])
+    _ = run(['concatenate', 'tests/test-files/test-firmware.bin', '--binary-user-config-filename',
+             'tests/test-files/test-storage-area.bin', '--new-filename', out_filename])
     with open(out_filename, 'rb') as out_file, open('tests/test-files/test-storage-area.bin', 'rb') as storage_file:
         out = out_file.read()
         storage = storage_file.read()
-    assert out[2088960:2097152] == storage
+    assert out[2064384:2097152] == storage
+
+
+def test_concatenate_invocation_json(tmpdir):
+    """Test that a normal invocation with a firmware and a JSON file works."""
+    out_filename = os.path.join(tmpdir, 'out.bin')
+    _ = run(['concatenate', '-P', 'tests/test-files/proto-files', 'tests/test-files/test-firmware.bin',
+             '--json-user-config-filename', 'tests/test-files/test-config.json', '--new-filename',
+             out_filename])
+    with open(out_filename, 'rb') as out_file, open('tests/test-files/test-binary-source-of-json-config.bin',
+                                                    'rb') as storage_file:
+        out = out_file.read()
+        storage = storage_file.read()
+    assert out[2093382:2097152] == storage
+
+
+def test_summarize_invocation(tmpdir):
+    """Test that we can get some summary information."""
+    result = run(['summarize-gp2040ce', '--filename', 'tests/test-files/test-firmware.bin'],
+                 capture_output=True, encoding='utf8')
+    assert 'detected GP2040-CE version:     v0.7.5' in result.stdout
 
 
 def test_storage_dump_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-storage', '-P', 'tests/test-files/proto-files', 'tests/test-files/test-storage-area.bin'],
+    result = run(['visualize-config', '-P', 'tests/test-files/proto-files',
+                  '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
-    assert 'boardVersion: "v0.7.2"' in result.stdout
+    assert 'boardVersion: "v0.7.5"' in result.stdout
 
 
 def test_debug_storage_dump_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-storage', '-d', '-P', 'tests/test-files/proto-files',
-                  'tests/test-files/test-storage-area.bin'],
+    result = run(['visualize-config', '-d', '-P', 'tests/test-files/proto-files',
+                  '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
-    assert 'boardVersion: "v0.7.2"' in result.stdout
-    assert 'length of content to look for footer in: 8192' in result.stderr
+    assert 'boardVersion: "v0.7.5"' in result.stdout
+    assert 'length of content to look for footer in: 32768' in result.stderr
 
 
 def test_storage_dump_json_invocation():
     """Test that a normal invocation against a dump works."""
-    result = run(['visualize-storage', '-P', 'tests/test-files/proto-files', '--json',
-                  'tests/test-files/test-storage-area.bin'],
+    result = run(['visualize-config', '-P', 'tests/test-files/proto-files', '--json',
+                  '--filename', 'tests/test-files/test-storage-area.bin'],
                  capture_output=True, encoding='utf8')
     to_dict = json.loads(result.stdout)
-    assert to_dict['boardVersion'] == 'v0.7.2'
+    assert to_dict['boardVersion'] == 'v0.7.5'

tests/test_gui.py

@@ -0,0 +1,280 @@
+"""Test the Textual GUI.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
+import os
+import sys
+import unittest.mock as mock
+
+import pytest
+from decorator import decorator
+from textual.widgets import Tree
+
+from gp2040ce_bintools import get_config_pb2
+from gp2040ce_bintools.gui import ConfigEditor
+from gp2040ce_bintools.storage import ConfigReadError, get_config, get_config_from_file
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+
+@decorator
+async def with_pb2s(test, *args, **kwargs):
+    """Wrap a test with precompiled pb2 files on the path."""
+    proto_path = os.path.join(HERE, 'test-files', 'pb2-files')
+    sys.path.append(proto_path)
+
+    await test(*args, **kwargs)
+
+    sys.path.pop()
+    del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_load_configs():
+    """Test a variety of ways the editor may get initialized."""
+    test_config_filename = os.path.join(HERE, 'test-files/test-config.bin')
+    empty_config = get_config_pb2().Config()
+    with open(test_config_filename, 'rb') as file_:
+        test_config_binary = file_.read()
+    test_config = get_config(test_config_binary)
+
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    assert app.config == test_config
+
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.binooooooo'), create_new=True)
+    assert app.config == empty_config
+
+    with pytest.raises(FileNotFoundError):
+        app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.binooooooo'))
+
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-firmware.bin'), create_new=True)
+    assert app.config == empty_config
+
+    with pytest.raises(ConfigReadError):
+        app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-firmware.bin'))
+
+    with mock.patch('gp2040ce_bintools.gui.get_bootsel_endpoints', return_value=(mock.MagicMock(), mock.MagicMock())):
+        with mock.patch('gp2040ce_bintools.gui.read', return_value=b'\x00'):
+            with pytest.raises(ConfigReadError):
+                app = ConfigEditor(usb=True)
+
+    with mock.patch('gp2040ce_bintools.gui.get_bootsel_endpoints', return_value=(mock.MagicMock(), mock.MagicMock())):
+        with mock.patch('gp2040ce_bintools.gui.read', return_value=b'\x00'):
+            app = ConfigEditor(usb=True, create_new=True)
+    assert app.config == empty_config
+
+    with mock.patch('gp2040ce_bintools.gui.get_bootsel_endpoints', return_value=(mock.MagicMock(), mock.MagicMock())):
+        with mock.patch('gp2040ce_bintools.gui.read', return_value=test_config_binary):
+            app = ConfigEditor(usb=True)
+    assert app.config == test_config
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_simple_tree_building():
+    """Test some basics of the config tree being built."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        check_node = pilot.app.query_one(Tree).root.children[3]
+        assert "boardVersion = 'v0.7.5'" in check_node.label
+        parent_config, field_descriptor, field_value = check_node.data
+        assert parent_config == pilot.app.config
+        assert field_descriptor == pilot.app.config.DESCRIPTOR.fields_by_name['boardVersion']
+        assert field_value == 'v0.7.5'
+        app.exit()
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_simple_toggle():
+    """Test that we can navigate a bit and toggle a bool."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        tree = pilot.app.query_one(Tree)
+        display_node = tree.root.children[5]
+        invert_node = display_node.children[10]
+
+        assert 'False' in invert_node.label
+        app._modify_node(invert_node)
+        assert 'True' in invert_node.label
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_simple_edit_via_input_field():
+    """Test that we can change an int via UI and see it reflected in the config."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        tree = pilot.app.query_one(Tree)
+        display_node = tree.root.children[5]
+        i2cspeed_node = display_node.children[6]
+        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 400000
+
+        tree.root.expand_all()
+        await pilot.wait_for_scheduled_animations()
+        tree.select_node(i2cspeed_node)
+        tree.action_select_cursor()
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Input#field-input')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.press('end', 'backspace', 'backspace', 'backspace', 'backspace', 'backspace', 'backspace', '5')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#confirm-button')
+        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 5
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_cancel_simple_edit_via_input_field():
+    """Test that we can cancel out of saving an int via UI and see it reflected in the config."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        tree = pilot.app.query_one(Tree)
+        display_node = tree.root.children[5]
+        i2cspeed_node = display_node.children[4]
+        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 400000
+
+        tree.root.expand_all()
+        await pilot.wait_for_scheduled_animations()
+        tree.select_node(i2cspeed_node)
+        tree.action_select_cursor()
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Input#field-input')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.press('backspace', 'backspace', 'backspace', 'backspace', 'backspace', 'backspace', '5')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#cancel-button')
+        assert pilot.app.config.displayOptions.deprecatedI2cSpeed == 400000
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_about():
+    """Test that we can bring up the about box."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        await pilot.press('?')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#ok-button')
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_simple_edit_via_input_field_enum():
+    """Test that we can change an enum via the UI and see it reflected in the config."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        tree = pilot.app.query_one(Tree)
+        gamepad_node = tree.root.children[7]
+        dpadmode_node = gamepad_node.children[0]
+        assert pilot.app.config.gamepadOptions.dpadMode == 0
+
+        tree.root.expand_all()
+        await pilot.wait_for_scheduled_animations()
+        tree.select_node(dpadmode_node)
+        tree.action_select_cursor()
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Select#field-input')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.press('down', 'down', 'enter')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#confirm-button')
+        assert pilot.app.config.gamepadOptions.dpadMode == 1
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_simple_edit_via_input_field_string():
+    """Test that we can change a string via the UI and see it reflected in the config."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        tree = pilot.app.query_one(Tree)
+        version_node = tree.root.children[3]
+        assert pilot.app.config.boardVersion == 'v0.7.5'
+
+        tree.select_node(version_node)
+        tree.action_select_cursor()
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Input#field-input')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.press('end', 'backspace', '-', 'h', 'i')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#confirm-button')
+        assert pilot.app.config.boardVersion == 'v0.7.-hi'
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_add_node_to_repeated():
+    """Test that we can navigate to an empty repeated and add a node."""
+    app = ConfigEditor(config_filename=os.path.join(HERE, 'test-files/test-config.bin'))
+    async with app.run_test() as pilot:
+        tree = pilot.app.query_one(Tree)
+        profile_node = tree.root.children[13]
+        altpinmappings_node = profile_node.children[0]
+
+        tree.root.expand_all()
+        await pilot.wait_for_scheduled_animations()
+        tree.select_node(altpinmappings_node)
+        await pilot.press('n')
+        newpinmappings_node = altpinmappings_node.children[0]
+        newpinmappings_node.expand()
+        await pilot.wait_for_scheduled_animations()
+        tree.select_node(newpinmappings_node)
+        b4_node = newpinmappings_node.children[3]
+        tree.select_node(b4_node)
+        tree.action_select_cursor()
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Input#field-input')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.press('end', 'backspace', 'backspace', 'backspace', 'backspace', 'backspace', 'backspace', '5')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#confirm-button')
+
+        assert pilot.app.config.profileOptions.deprecatedAlternativePinMappings[0].pinButtonB4 == 5
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_save(config_binary, tmp_path):
+    """Test that the tree builds and things are kind of where they should be."""
+    new_filename = os.path.join(tmp_path, 'config-copy.bin')
+    with open(new_filename, 'wb') as file:
+        file.write(config_binary)
+
+    app = ConfigEditor(config_filename=new_filename)
+    async with app.run_test() as pilot:
+        pilot.app.config.boardVersion = 'v0.7.5-bss-wuz-here'
+        await pilot.press('s')
+
+    config = get_config_from_file(new_filename)
+    assert config.boardVersion == 'v0.7.5-bss-wuz-here'
+
+
+@pytest.mark.asyncio
+@with_pb2s
+async def test_save_as(config_binary, tmp_path):
+    """Test that we can save to a new file."""
+    filename = os.path.join(tmp_path, 'config-original.bin')
+    with open(filename, 'wb') as file:
+        file.write(config_binary)
+    original_config = get_config(config_binary)
+
+    app = ConfigEditor(config_filename=filename)
+    async with app.run_test() as pilot:
+        await pilot.press('a')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Input#field-input')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.press('/', 't', 'm', 'p', '/', 'g', 'p', 't', 'e', 's', 't')
+        await pilot.wait_for_scheduled_animations()
+        await pilot.click('Button#confirm-button')
+
+    with open('/tmp/gptest', 'rb') as new_file:
+        test_config_binary = new_file.read()
+    test_config = get_config(test_config_binary)
+    assert original_config.boardVersion == test_config.boardVersion

tests/test_package.py

@@ -1,45 +1,74 @@
-"""Test high level package capabilities."""
+"""Test high level package capabilities.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
 import os
 import sys
 
 import pytest
+from decorator import decorator
 
 from gp2040ce_bintools import get_config_pb2
 
 HERE = os.path.dirname(os.path.abspath(__file__))
 
 
-def test_get_config_pb2_compile():
-    """Without any precompiled files on the path, test we can read proto files and compile them."""
-    # append to path as -P would
-    proto_path = os.path.join(HERE, 'test-files', 'proto-files')
-    sys.path.append(proto_path)
-
-    # let grpc tools compile the proto files on demand and give us the module
-    config_pb2 = get_config_pb2()
-    _ = config_pb2.Config()
-
-    # clean up the path and unload config_pb2
-    sys.path.pop()
-    sys.path.pop()
-    del sys.modules['config_pb2']
-
-
-def test_get_config_pb2_exception():
-    """Without any precompiled files or proto files on the path, test we raise an exception."""
-    with pytest.raises(ModuleNotFoundError):
-        _ = get_config_pb2()
-
-
-def test_get_config_pb2_precompile():
-    """Test we can import precompiled protobuf files."""
+@decorator
+def with_pb2s(test, *args, **kwargs):
+    """Wrap a test with precompiled pb2 files on the path."""
     proto_path = os.path.join(HERE, 'test-files', 'pb2-files')
     sys.path.append(proto_path)
 
-    # let grpc tools import the proto files normally
+    test(*args, **kwargs)
+
+    sys.path.pop()
+    del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
+
+
+@decorator
+def with_protos(test, *args, **kwargs):
+    """Wrap a test with .proto files on the path."""
+    proto_path = os.path.join(HERE, 'test-files', 'proto-files')
+    sys.path.append(proto_path)
+
+    test(*args, **kwargs)
+
+    sys.path.pop()
+    del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
+
+
+@with_pb2s
+def test_get_config_pb2_precompiled():
+    """With precompiled files on the path, test we can read and use them."""
+    # get the module from the provided files
     config_pb2 = get_config_pb2()
     _ = config_pb2.Config()
 
-    # clean up the path and unload config_pb2
-    sys.path.pop()
+
+def test_get_config_pb2_exception():
+    """Test that we fail if no config .proto files are available."""
+    with pytest.raises(RuntimeError):
+        _ = get_config_pb2()
+
+
+def test_get_config_pb2_shipped_config_files():
+    """Without any precompiled files or proto files on the path, test we DO NOT raise an exception."""
+    # use the shipped .proto files to generate the config
+    config_pb2 = get_config_pb2(with_fallback=True)
+    _ = config_pb2.Config()
     del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
+
+
+@with_protos
+def test_get_config_pb2_compile():
+    """Without any precompiled files on the path, test we can read proto files and compile them."""
+    # let grpc tools compile the proto files on demand and give us the module
+    config_pb2 = get_config_pb2()
+    _ = config_pb2.Config()

tests/test_rp2040.py

@@ -0,0 +1,251 @@
+"""Test operations for interfacing directly with a Pico.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
+import os
+import struct
+import sys
+import unittest.mock as mock
+from array import array
+
+import pytest
+from decorator import decorator
+
+import gp2040ce_bintools.rp2040 as rp2040
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+
+@decorator
+def with_pb2s(test, *args, **kwargs):
+    """Wrap a test with precompiled pb2 files on the path."""
+    proto_path = os.path.join(HERE, 'test-files', 'pb2-files')
+    sys.path.append(proto_path)
+
+    test(*args, **kwargs)
+
+    sys.path.pop()
+    del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
+
+
+def test_get_bootsel_endpoints():
+    """Test our expected method of finding the BOOTSEL mode Pico board."""
+    mock_device = mock.MagicMock(name='mock_device')
+    mock_device.is_kernel_driver_active.return_value = False
+    mock_configuration = mock.MagicMock(name='mock_configuration')
+    mock_device.get_active_configuration.return_value = mock_configuration
+    mock_interface = mock.MagicMock(name='mock_interface')
+    with mock.patch('usb.core.find', return_value=mock_device) as mock_find:
+        with mock.patch('usb.util.find_descriptor', return_value=mock_interface) as mock_find_descriptor:
+            _, _ = rp2040.get_bootsel_endpoints()
+
+    mock_find.assert_called_with(idVendor=rp2040.PICO_VENDOR, idProduct=rp2040.PICO_PRODUCT)
+    mock_device.is_kernel_driver_active.assert_called_with(0)
+    mock_device.detach_kernel_driver.assert_not_called()
+    mock_device.get_active_configuration.assert_called_once()
+    assert mock_find_descriptor.call_args_list[0].args[0] == mock_configuration
+    assert mock_find_descriptor.call_args_list[1].args[0] == mock_interface
+    assert mock_find_descriptor.call_args_list[2].args[0] == mock_interface
+
+
+def test_get_bootsel_endpoints_with_kernel_disconnect():
+    """Test our expected method of finding the BOOTSEL mode Pico board."""
+    mock_device = mock.MagicMock(name='mock_device')
+    mock_device.is_kernel_driver_active.return_value = True
+    mock_configuration = mock.MagicMock(name='mock_configuration')
+    mock_device.get_active_configuration.return_value = mock_configuration
+    mock_interface = mock.MagicMock(name='mock_interface')
+    with mock.patch('usb.core.find', return_value=mock_device) as mock_find:
+        with mock.patch('usb.util.find_descriptor', return_value=mock_interface) as mock_find_descriptor:
+            _, _ = rp2040.get_bootsel_endpoints()
+
+    mock_find.assert_called_with(idVendor=rp2040.PICO_VENDOR, idProduct=rp2040.PICO_PRODUCT)
+    mock_device.is_kernel_driver_active.assert_called_with(0)
+    mock_device.detach_kernel_driver.assert_called_with(0)
+    mock_device.get_active_configuration.assert_called_once()
+    assert mock_find_descriptor.call_args_list[0].args[0] == mock_configuration
+    assert mock_find_descriptor.call_args_list[1].args[0] == mock_interface
+    assert mock_find_descriptor.call_args_list[2].args[0] == mock_interface
+
+
+def test_exclusive_access():
+    """Test that we can get exclusive access to a BOOTSEL board."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    rp2040.exclusive_access(end_out, end_in)
+
+    payload = struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 1)
+    end_out.write.assert_called_with(payload)
+    end_in.read.assert_called_once()
+
+    end_out.reset_mock()
+    end_in.reset_mock()
+    rp2040.exclusive_access(end_out, end_in, is_exclusive=False)
+
+    payload = struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 0)
+    end_out.write.assert_called_with(payload)
+    end_in.read.assert_called_once()
+
+
+def test_exit_xip():
+    """Test that we can exit XIP on a BOOTSEL board."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    rp2040.exit_xip(end_out, end_in)
+
+    payload = struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)
+    end_out.write.assert_called_with(payload)
+    end_in.read.assert_called_once()
+
+
+def test_erase():
+    """Test that we can send a command to erase a section of memory."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    rp2040.erase(end_out, end_in, 0x101FC000, 8192)
+
+    payload = struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x3, 8, 0, 0x101FC000, 8192)
+    end_out.write.assert_called_with(payload)
+    end_in.read.assert_called_once()
+
+
+def test_read():
+    """Test that we can read a memory of a BOOTSEL board in a variety of conditions."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    end_in.read.return_value = array('B', b'\x11' * 256)
+    content = rp2040.read(end_out, end_in, 0x101FC000, 256)
+
+    expected_writes = [
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 1)),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x84, 8, 256, 0x101FC000, 256)),
+        mock.call(b'\xc0'),
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 0)),
+    ]
+    end_out.write.assert_has_calls(expected_writes)
+    assert end_in.read.call_count == 4
+    assert len(content) == 256
+
+
+def test_read_shorter_than_chunk():
+    """Test that we can read a memory of a BOOTSEL board in a variety of conditions."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    end_in.read.return_value = array('B', b'\x11' * 256)
+    content = rp2040.read(end_out, end_in, 0x101FC000, 128)
+
+    expected_writes = [
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 1)),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x84, 8, 256, 0x101FC000, 256)),
+        mock.call(b'\xc0'),
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 0)),
+    ]
+    end_out.write.assert_has_calls(expected_writes)
+    assert end_in.read.call_count == 4
+    assert len(content) == 128
+
+
+def test_read_bigger_than_chunk():
+    """Test that we can read a memory of a BOOTSEL board in a variety of conditions."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    end_in.read.return_value = array('B', b'\x11' * 256)
+    content = rp2040.read(end_out, end_in, 0x101FC000, 512)
+
+    expected_writes = [
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 1)),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x84, 8, 256, 0x101FC000, 256)),
+        mock.call(b'\xc0'),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x84, 8, 256, 0x101FC000+256, 256)),
+        mock.call(b'\xc0'),
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 0)),
+    ]
+    end_out.write.assert_has_calls(expected_writes)
+    assert end_in.read.call_count == 6
+    assert len(content) == 512
+
+
+def test_reboot():
+    """Test that we can reboot a BOOTSEL board."""
+    end_out = mock.MagicMock()
+    rp2040.reboot(end_out)
+
+    payload = struct.pack('<LLBBxxLLLL4x', 0x431fd10b, 1, 0x2, 12, 0, 0, 0x20042000, 500)
+    end_out.write.assert_called_with(payload)
+
+
+def test_write():
+    """Test that we can write to a board in BOOTSEL mode."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    _ = rp2040.write(end_out, end_in, 0x101FC000, b'\x00\x01\x02\x03')
+
+    expected_writes = [
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 1)),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x3, 8, 0, 0x101FC000, 4)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x5, 8, 4, 0x101FC000, 4)),
+        mock.call(b'\x00\x01\x02\x03'),
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 0)),
+    ]
+    end_out.write.assert_has_calls(expected_writes)
+    assert end_in.read.call_count == 5
+
+
+def test_write_chunked():
+    """Test that we can write to a board in BOOTSEL mode."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    payload = bytearray(b'\x00\x01\x02\x03' * 1024)
+    _ = rp2040.write(end_out, end_in, 0x10100000, payload * 2)
+
+    expected_writes = [
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 1)),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x3, 8, 0, 0x10100000, 4096)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x5, 8, 4096, 0x10100000, 4096)),
+        mock.call(bytes(payload)),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x3, 8, 0, 0x10100000 + 4096, 4096)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x5, 8, 4096, 0x10100000 + 4096, 4096)),
+        mock.call(bytes(payload)),
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 0)),
+    ]
+    end_out.write.assert_has_calls(expected_writes)
+    assert end_in.read.call_count == 8
+
+
+def test_misaligned_write():
+    """Test that we can't write to a board at invalid memory addresses."""
+    end_out, end_in = mock.MagicMock(), mock.MagicMock()
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE001, b'\x00\x01\x02\x03')
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE008, b'\x00\x01\x02\x03')
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE010, b'\x00\x01\x02\x03')
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE020, b'\x00\x01\x02\x03')
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE040, b'\x00\x01\x02\x03')
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE080, b'\x00\x01\x02\x03')
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE0FF, b'\x00\x01\x02\x03')
+
+    # 256 byte alignment is what is desired, but see comments around there for
+    # why only 4096 seems to work right...
+    with pytest.raises(rp2040.RP2040AlignmentError):
+        _ = rp2040.write(end_out, end_in, 0x101FE100, b'\x00\x01\x02\x03')
+
+    _ = rp2040.write(end_out, end_in, 0x101FF000, b'\x00\x01\x02\x03')
+
+    expected_writes = [
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 1)),
+        mock.call(struct.pack('<LLBBxxL16x', 0x431fd10b, 1, 0x6, 0, 0)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x3, 8, 0, 0x101FF000, 4)),
+        mock.call(struct.pack('<LLBBxxLLL8x', 0x431fd10b, 1, 0x5, 8, 4, 0x101FF000, 4)),
+        mock.call(b'\x00\x01\x02\x03'),
+        mock.call(struct.pack('<LLBBxxLL12x', 0x431fd10b, 1, 0x1, 1, 0, 0)),
+    ]
+    end_out.write.assert_has_calls(expected_writes)
+    assert end_in.read.call_count == 5

tests/test_storage.py

@@ -1,11 +1,18 @@
-"""Unit tests for the storage module."""
+"""Unit tests for the storage module.
+
+SPDX-FileCopyrightText: © 2023 Brian S. Stephan <bss@incorporeal.org>
+SPDX-License-Identifier: GPL-3.0-or-later
+"""
+import math
 import os
 import sys
+import unittest.mock as mock
 
 import pytest
 from decorator import decorator
 
 import gp2040ce_bintools.storage as storage
+from gp2040ce_bintools.builder import concatenate_firmware_and_storage_files
 
 HERE = os.path.dirname(os.path.abspath(__file__))
 
@@ -20,13 +27,15 @@ def with_pb2s(test, *args, **kwargs):
 
     sys.path.pop()
     del sys.modules['config_pb2']
+    del sys.modules['enums_pb2']
+    del sys.modules['nanopb_pb2']
 
 
 def test_config_footer(storage_dump):
     """Test that a config footer is identified as expected."""
     size, crc, magic = storage.get_config_footer(storage_dump)
-    assert size == 2032
-    assert crc == 3799109329
+    assert size == 3309
+    assert crc == 2661279683
     assert magic == '0x65e3f1d2'
 
 
@@ -45,7 +54,7 @@ def test_config_footer_too_small(storage_dump):
 def test_whole_board_too_small(whole_board_dump):
     """Test that a storage section isn't detected if the size is too small to contain where it should be."""
     with pytest.raises(storage.ConfigLengthError):
-        _, _, _ = storage.get_storage_section(whole_board_dump[-100000:])
+        _, _, _ = storage.get_user_storage_section(whole_board_dump[-100000:])
 
 
 def test_config_footer_bad_magic(storage_dump):
@@ -56,10 +65,12 @@ def test_config_footer_bad_magic(storage_dump):
         _, _, _ = storage.get_config_footer(unmagical)
 
 
-def test_config_fails_without_pb2s(storage_dump):
-    """Test that we need the config_pb2 to exist/be compiled for reading the config to work."""
-    with pytest.raises(ModuleNotFoundError):
-        _ = storage.get_config(storage_dump)
+def test_config_footer_bad_crc(storage_dump):
+    """Test that a config footer isn't detected if the CRC checksums don't match."""
+    corrupt = bytearray(storage_dump)
+    corrupt[-50:-40] = bytearray(0 * 10)
+    with pytest.raises(storage.ConfigCrcError):
+        _, _, _ = storage.get_config_footer(corrupt)
 
 
 @with_pb2s
@@ -67,7 +78,7 @@ def test_get_config_from_file_storage_dump():
     """Test that we can open a storage dump file and find its config."""
     filename = os.path.join(HERE, 'test-files', 'test-storage-area.bin')
     config = storage.get_config_from_file(filename)
-    assert config.boardVersion == 'v0.7.2'
+    assert config.boardVersion == 'v0.7.5'
     assert config.addonOptions.bootselButtonOptions.enabled is False
     assert config.addonOptions.ps4Options.enabled is False
 
@@ -77,33 +88,222 @@ def test_get_config_from_file_whole_board_dump():
     """Test that we can open a storage dump file and find its config."""
     filename = os.path.join(HERE, 'test-files', 'test-whole-board.bin')
     config = storage.get_config_from_file(filename, whole_board=True)
-    assert config.boardVersion == 'v0.7.2'
+    assert config.boardVersion == 'v0.7.5'
     assert config.addonOptions.bootselButtonOptions.enabled is False
 
 
+@with_pb2s
+def test_get_board_config_from_file_whole_board_dump():
+    """Test that we can open a storage dump file and find its config."""
+    filename = os.path.join(HERE, 'test-files', 'test-whole-board-with-board-config.bin')
+    config = storage.get_config_from_file(filename, whole_board=True, board_config=True)
+    assert config.boardVersion == 'v0.7.8'
+    assert config.addonOptions.bootselButtonOptions.enabled is False
+
+
+@with_pb2s
+def test_get_board_config_from_json_file():
+    """Test that we can open a JSON file and parse the config."""
+    filename = os.path.join(HERE, 'test-files', 'test-config.json')
+    config = storage.get_config_from_file(filename, whole_board=True, board_config=True)
+    assert config.boardVersion == 'v0.7.6-15-g71f4512'
+    assert config.addonOptions.bootselButtonOptions.enabled is False
+
+
+@with_pb2s
+def test_get_config_from_file_file_not_fonud_ok():
+    """If we allow opening a file that doesn't exist (e.g. for the editor), check we get an empty config."""
+    filename = os.path.join(HERE, 'test-files', 'nope.bin')
+    config = storage.get_config_from_file(filename, allow_no_file=True)
+    assert config.boardVersion == ''
+
+
+def test_get_config_from_file_file_not_fonud_raise():
+    """If we don't allow opening a file that doesn't exist (e.g. for the editor), check we get an error."""
+    filename = os.path.join(HERE, 'test-files', 'nope.bin')
+    with pytest.raises(FileNotFoundError):
+        _ = storage.get_config_from_file(filename)
+
+
 @with_pb2s
 def test_config_parses(storage_dump):
     """Test that we need the config_pb2 to exist/be compiled for reading the config to work."""
     config = storage.get_config(storage_dump)
-    assert config.boardVersion == 'v0.7.2'
-    assert config.hotkeyOptions.hotkeyF1Up.dpadMask == 1
+    assert config.boardVersion == 'v0.7.5'
+    assert config.hotkeyOptions.hotkey01.dpadMask == 0
+    assert config.hotkeyOptions.hotkey02.dpadMask == 1
 
 
 @with_pb2s
 def test_config_from_whole_board_parses(whole_board_dump):
     """Test that we can read in a whole board and still find the config section."""
-    config = storage.get_config(storage.get_storage_section(whole_board_dump))
-    assert config.boardVersion == 'v0.7.2'
-    assert config.hotkeyOptions.hotkeyF1Up.dpadMask == 1
+    config = storage.get_config(storage.get_user_storage_section(whole_board_dump))
+    assert config.boardVersion == 'v0.7.5'
+    assert config.hotkeyOptions.hotkey01.dpadMask == 0
+    assert config.hotkeyOptions.hotkey02.dpadMask == 1
+
+
+def test_convert_binary_to_uf2(whole_board_with_board_config_dump):
+    """Do some sanity checks in the attempt to convert a binary to a UF2."""
+    uf2 = storage.convert_binary_to_uf2([{0, whole_board_with_board_config_dump}])
+    assert len(uf2) == 4194304                              # binary is 8192 256 byte chunks, UF2 is 512 b per chunk
+    assert uf2[0:4] == b'\x55\x46\x32\x0a' == b'UF2\n'      # proper magic
+    assert uf2[8:12] == bytearray(b'\x00\x20\x00\x00')      # family ID set
+    assert uf2[524:528] == bytearray(b'\x00\x01\x00\x10')   # address to write the second chunk
+
+
+def test_convert_unaligned_binary_to_uf2(firmware_binary):
+    """Do some sanity checks in the attempt to convert a binary to a UF2."""
+    uf2 = storage.convert_binary_to_uf2([{0, firmware_binary}])
+    assert len(uf2) == math.ceil(len(firmware_binary)/256) * 512    # 256 byte complete/partial chunks -> 512 b chunks
+    assert uf2[0:4] == b'\x55\x46\x32\x0a' == b'UF2\n'      # proper magic
+    assert uf2[8:12] == bytearray(b'\x00\x20\x00\x00')      # family ID set
+    assert uf2[524:528] == bytearray(b'\x00\x01\x00\x10')   # address to write the second chunk
+
+
+def test_convert_binary_to_uf2_with_offsets(whole_board_with_board_config_dump):
+    """Do some sanity checks in the attempt to convert a binary to a UF2."""
+    uf2 = storage.convert_binary_to_uf2([{storage.USER_CONFIG_BINARY_LOCATION, whole_board_with_board_config_dump}])
+    assert len(uf2) == 4194304                              # binary is 8192 256 byte chunks, UF2 is 512 b per chunk
+    assert uf2[0:4] == b'\x55\x46\x32\x0a' == b'UF2\n'      # proper magic
+    assert uf2[8:12] == bytearray(b'\x00\x20\x00\x00')      # family ID set
+    assert uf2[524:528] == bytearray(b'\x00\x81\x1f\x10')   # address to write the second chunk
+
+
+def test_convert_binary_to_uf2_to_binary(whole_board_with_board_config_dump):
+    """Do some sanity checks in the attempt to convert a binary to a UF2."""
+    uf2 = storage.convert_binary_to_uf2([{0, whole_board_with_board_config_dump}])
+    binary = storage.convert_uf2_to_binary(uf2)
+    assert len(binary) == 2097152
+    assert whole_board_with_board_config_dump == binary
+
+
+def test_malformed_uf2(whole_board_with_board_config_dump):
+    """Check that we expect a properly-formed UF2."""
+    uf2 = storage.convert_binary_to_uf2([{0, whole_board_with_board_config_dump}])
+
+    # truncated UF2 --- byte mismatch
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2[:-4])
+
+    # truncated uf2 --- counter is wrong
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2[512:])
+
+    # truncated uf2 --- total count is wrong
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2[:-512])
+
+    # malformed UF2 --- counter jumps in the middle, suggests total blocks is wrong
+    with pytest.raises(ValueError):
+        storage.convert_uf2_to_binary(uf2 + uf2)
+
+
+def test_read_created_uf2(tmp_path, firmware_binary, config_binary):
+    """Test that we read a UF2 with disjoint segments."""
+    tmp_file = os.path.join(tmp_path, 'concat.uf2')
+    firmware_file = os.path.join(HERE, 'test-files', 'test-firmware.bin')
+    config_file = os.path.join(HERE, 'test-files', 'test-config.bin')
+    concatenate_firmware_and_storage_files(firmware_file, binary_board_config_filename=config_file,
+                                           binary_user_config_filename=config_file,
+                                           combined_filename=tmp_file)
+    with open(tmp_file, 'rb') as file:
+        content = file.read()
+    assert len(content) == (math.ceil(len(firmware_binary)/256) * 512 +
+                            math.ceil(storage.STORAGE_SIZE/256) * 512 * 2)
+
+    binary = storage.convert_uf2_to_binary(content)
+    # the converted binary should be aligned properly and of the right size
+    assert len(binary) == 2 * 1024 * 1024
+    assert binary[-32768-4:-32768] == storage.FOOTER_MAGIC
+    assert binary[-4:] == storage.FOOTER_MAGIC
+    user_storage = storage.get_user_storage_section(binary)
+    footer_size, _, _ = storage.get_config_footer(user_storage)
+    assert footer_size == 3309
+
+
+def test_cant_read_out_of_order_uf2():
+    """Test that we currently raise an exception at out of order UF2s until we fix it."""
+    uf2 = storage.convert_binary_to_uf2([(0x1000, b'\x11'), (0, b'\x11')])
+    with pytest.raises(NotImplementedError):
+        storage.convert_uf2_to_binary(uf2)
+
+
+@with_pb2s
+def test_serialize_config_with_footer(storage_dump, config_binary):
+    """Test that reserializing a read in config matches the original.
+
+    Note that this isn't going to produce an *identical* result, because new message fields
+    may have default values that get saved in the reserialized binary, so we can still only test
+    some particular parts. But it should work.
+    """
+    config = storage.get_config(storage_dump)
+    assert config.boardVersion == 'v0.7.5'
+    reserialized = storage.serialize_config_with_footer(config)
+    assert storage_dump[-4:] == reserialized[-4:]
+
+
+@with_pb2s
+def test_serialize_modified_config_with_footer(storage_dump):
+    """Test that we can serialize a modified config."""
+    config = storage.get_config(storage_dump)
+    config.boardVersion = 'v0.7.5-cool'
+    serialized = storage.serialize_config_with_footer(config)
+    config_size, _, _ = storage.get_config_footer(serialized)
+    assert config_size == config.ByteSize()
+    assert len(serialized) == config_size + 12
 
 
 def test_pad_config_to_storage(config_binary):
     """Test that we can properly pad a config section to the correct storage section size."""
     storage_section = storage.pad_config_to_storage_size(config_binary)
-    assert len(storage_section) == 8192
+    assert len(storage_section) == 32768
 
 
 def test_pad_config_to_storage_raises(config_binary):
     """Test that we raise an exception if the config is bigger than the storage section."""
     with pytest.raises(storage.ConfigLengthError):
-        _ = storage.pad_config_to_storage_size(config_binary * 5)
+        _ = storage.pad_config_to_storage_size(config_binary * 10)
+
+
+@with_pb2s
+def test_get_board_config_from_usb(config_binary):
+    """Test we attempt to read from the proper location over USB."""
+    mock_out = mock.MagicMock()
+    mock_out.device.idVendor = 0xbeef
+    mock_out.device.idProduct = 0xcafe
+    mock_out.device.bus = 1
+    mock_out.device.address = 2
+    mock_in = mock.MagicMock()
+    with mock.patch('gp2040ce_bintools.storage.get_bootsel_endpoints', return_value=(mock_out, mock_in)) as mock_get:
+        with mock.patch('gp2040ce_bintools.storage.read', return_value=config_binary) as mock_read:
+            config, _, _ = storage.get_board_config_from_usb()
+
+    mock_get.assert_called_once()
+    mock_read.assert_called_with(mock_out, mock_in, 0x101F0000, 32768)
+    assert config == storage.get_config(config_binary)
+
+
+@with_pb2s
+def test_get_user_config_from_usb(config_binary):
+    """Test we attempt to read from the proper location over USB."""
+    mock_out = mock.MagicMock()
+    mock_out.device.idVendor = 0xbeef
+    mock_out.device.idProduct = 0xcafe
+    mock_out.device.bus = 1
+    mock_out.device.address = 2
+    mock_in = mock.MagicMock()
+    with mock.patch('gp2040ce_bintools.storage.get_bootsel_endpoints', return_value=(mock_out, mock_in)) as mock_get:
+        with mock.patch('gp2040ce_bintools.storage.read', return_value=config_binary) as mock_read:
+            config, _, _ = storage.get_user_config_from_usb()
+
+    mock_get.assert_called_once()
+    mock_read.assert_called_with(mock_out, mock_in, 0x101F8000, 32768)
+    assert config == storage.get_config(config_binary)
+
+
+@with_pb2s
+def test_json_config_parses(config_json):
+    """Test that we can import a JSON config into a message."""
+    config = storage.get_config_from_json(config_json)
+    assert config.boardVersion == 'v0.7.6-15-g71f4512'

tox.ini

@@ -5,7 +5,7 @@
 
 [tox]
 isolated_build = true
-envlist = begin,py39,py310,py311,coverage,bandit,lint
+envlist = begin,py39,py310,py311,py312,coverage,bandit,lint,reuse
 
 [testenv]
 allow_externals = pytest, coverage
@@ -33,6 +33,11 @@ commands =
 commands =
     pytest --cov-append --cov={envsitepackagesdir}/gp2040ce_bintools/ --cov-branch
 
+[testenv:py312]
+# run pytest with coverage
+commands =
+    pytest --cov-append --cov={envsitepackagesdir}/gp2040ce_bintools/ --cov-branch
+
 [testenv:coverage]
 # report on coverage runs from above
 skip_install = true
@@ -57,6 +62,11 @@ commands =
     mypy gp2040ce_bintools
     - flake8 --disable-noqa --ignore= --select=E,W,F,C,D,A,G,B,I,T,M,DUO
 
+[testenv:reuse]
+# check license documentation
+commands =
+    reuse lint
+
 [coverage:paths]
 source =
     ./
@@ -67,3 +77,4 @@ branch = True
 
 omit =
     **/_version.py
+    **/proto_snapshot/*