use werkzeug safe_join to sanitize the requested path

no tests changed, so my implementation might have been good, but let's
use the provided check

CONTRIBUTING.md

@@ -0,0 +1,76 @@
+# How to Contribute
+
+incorporeal-cms is a personal project seeking to implement a simpler, cleaner form of what would
+commonly be called a "CMS". I appreciate any help in making it better.
+
+incorporeal-cms is made available under the GNU Affero General Public License version 3, or any
+later version.
+
+## Opening Issues
+
+Issues should be posted to my Gitea instance at
+<https://git.incorporeal.org/bss/incorporeal-cms/issues>. I'm not too picky about format, but I
+recommend starting the title with "Improvement:", "Bug:", or similar, so I can do a high level of
+prioritization.
+
+## Contributions
+
+I don't expect contributors to sign up for my personal Gitea in order to send contributions, but it
+of course makes it easier. If you wish to go this route, please sign up at
+<https://git.incorporeal.org/bss/incorporeal-cms> and fork the project. People planning on
+contributing often are also welcome to request access to the project directly.
+
+Otherwise, contact me via any means you know to reach me at, or <bss@incorporeal.org>, to discuss
+your change and to tell me how to pull your changes.
+
+### Guidelines for Patches, etc.
+
+* Cloning
+    * Clone the project. I would advise using a pull-based workflow where I have access to the hosted
+      repository --- using my Gitea, cloning to a public GitHub, etc. --- rather than doing this over
+      email, but that works too if we must.
+    * Make your contributions in a new branch, generally off of `master`.
+    * Send me a pull request when you're ready, and we'll go through a code review.
+* Code:
+    * Keep in mind that I strive for simplicity in the software. It serves files and renders
+      Markdown, that's pretty much it. Features around that function are good; otherwise, I need
+      convincing.
+    * Follow the style precedent set in the code. Do **not** use Black, or otherwise reformat existing
+      code. I like it the way it is and don't need a militant tool making bad decisions about what is
+      readable.
+    * `tox` should run cleanly, of course.
+    * Almost any change should include unit tests, and also functional tests if they provide a feature
+      to the CMS functionality. For defects, include unit tests that fail on the unfixed codebase, so I
+      know exactly what's happening.
+* Commits:
+    * Squash tiny commits if you'd like. I prefer commits that make one atomic conceptual change
+      that doesn't affect the rest of the code, assembling multiple of those commits into larger
+      changes.
+    * Follow something like [Chris Beams's post](https://chris.beams.io/posts/git-commit/) on
+      formatting a good commit message.
+    * Please make sure your Author contact information is stable, in case I need to reach you.
+    * Consider cryptographically signing (`git commit -S`) your commits.
+
+### Custody of Contributions
+
+I do not request the copyright of contributions be assigned to me or to the project, and I require no
+provision that I be allowed to relicense your contributions. My personal oath is to maintain
+inbound=outbound in my open source projects, and the expectation is authors are responsible for their
+contributions.
+
+I am following the *spirit* of the [Developer Certificate of Origin](https://developercertificate.org/),
+but in a simplified fashion:
+
+By making a contribution to this project, you certify that:
+
+1. The contribution was created by you and you have the right to submit it under the open source license
+   indicated in the LICENSE file; or
+2. The contribution is based upon previous work that is covered under an appropriate open source license
+   compatible with the license indicated in the LICENSE file, and you have the right to contribute that
+   work with or without modifications, under the terms of that same open source license; or
+3. The contribution was provided directly to you by some other person who certified points 1, 2, or 3, and
+   you have not modified it.
+
+In the event of point 3, your commit **must** include the Signed-off-by line(s) as a chain of custody,
+via `git commit -s`. For points 1 and 2, your commit with accurate Author information doubles as direct
+custody.

README.md

@@ -1,3 +1,66 @@
 # incorporeal-cms
 
-Software that makes incorporeal.org go.
+Software that makes simple Markdown content go.
+
+## Installation and Usage
+
+I recommend getting a release from <https://git.incorporeal.org/bss/incorporeal-cms/releases> and
+installing the Python package in a virtualenv. Something like the following should suffice:
+
+```
+% cd ~/site
+% virtualenv --python=python3.8 env-py3.8
+% source env-py3.8/bin/activate
+% pip install -U pip
+% pip install ~/incorporeal_cms-1.3.0-py3-none-any.whl
+% pip install -U gunicorn
+% gunicorn -w 5 -t 60 -b 127.0.0.1:10000 --reload 'incorporealcms:create_app()'
+```
+
+This will get the CMS up and running, and listening on the specified port. The application is
+further configured within `env-py3.8/var/incorporealcms-instance/config.py`, and content is served
+out of `env-py3.8/var/incorporealcms-instance/pages/`.
+
+## Serving a Site
+
+Put content inside `env-py3.8/var/incorporealcms-instance/pages/` and go.
+
+* Markdown files (ending in `.md`) are rendered via Python-Markdown if they are accessed without the
+  suffix (i.e., `post.md` should be referred to as `/post` to get it to render as Markdown.
+* Directory paths (e.g. `/dir/`) can be rendered with a `/dir/index.md` file.
+* Symlinks to files are treated as redirects to the destination content.
+* Request paths with file suffixes are not rendered and served directly, so images, etc., can be
+  referenced naturally, and even the unrendered Markdown can be served as a text file via e.g.
+  `/post.md`.
+
+Care is taken to not serve content above the `pages/` dir, even via symlink.
+
+## Configuration
+
+I've tried to keep the software agnostic to my personal domains, logos, etc. There are some settings
+you are probably interested in tweaking, by specifying new values in
+`incorporealcms-instance/config.py`:
+
+* `TITLE_SUFFIX` is appended to the title of every page, separated from other title content by a
+  dash.
+* `CONTACT_EMAIL` is referred to in error templates.
+* `FAVICON` supplies the image used in browser tabs and that kind of thing.
+
+If I missed anything, please let me know.
+
+## Author and Licensing
+
+Written by and copyright Brian S. Stephan (bss@incorporeal.org).
+
+This program is free software: you can redistribute it and/or modify
+it under the terms of the GNU Affero General Public License as published
+by the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU Affero General Public License for more details.
+
+You should have received a copy of the GNU Affero General Public License
+along with this program.  If not, see <https://www.gnu.org/licenses/>.

incorporealcms/__init__.py

@@ -3,8 +3,7 @@ import logging
 import os
 from logging.config import dictConfig
 
-import markdown
+from flask import Flask, request
-from flask import Flask, request, send_from_directory
 
 from ._version import get_versions
 
@@ -32,10 +31,6 @@ def create_app(instance_path=None, test_config=None):
 
     logger.debug("instance path: %s", app.instance_path)
 
-    # initialize markdown parser from config, but include
-    # extensions our app depends on, like the meta extension
-    app.config['md'] = markdown.Markdown(extensions=app.config['MARKDOWN_EXTENSIONS'] + ['meta'])
-
     @app.before_request
     def log_request():
         logger.info("REQUEST:  %s %s", request.method, request.path)
@@ -45,12 +40,11 @@ def create_app(instance_path=None, test_config=None):
         logger.info("RESPONSE: %s %s: %s", request.method, request.path, response.status)
         return response
 
-    @app.route(f'/{app.config["MEDIA_DIR"]}/<path:filename>')
+    from . import error_pages, pages, static
-    def media_files(filename):
-        return send_from_directory(os.path.join(app.instance_path, app.config['MEDIA_DIR']),
-                                   filename)
-
-    from . import pages
     app.register_blueprint(pages.bp)
+    app.register_blueprint(static.bp)
+    app.register_error_handler(400, error_pages.bad_request)
+    app.register_error_handler(404, error_pages.page_not_found)
+    app.register_error_handler(500, error_pages.internal_server_error)
 
     return app

incorporealcms/config.py

@@ -32,13 +32,32 @@ class Config(object):
         },
     }
 
-    MARKDOWN_EXTENSIONS = ['meta', 'tables']
+    MARKDOWN_EXTENSIONS = ['extra', 'incorporealcms.mdx.figures', 'sane_lists', 'smarty', 'toc']
-
+    MARKDOWN_EXTENSION_CONFIGS = {
-    PAGE_STYLES = {
+        'extra': {
-        'DEFAULT': 'css/light.css',
+            'footnotes': {
-        'dark': 'css/dark.css',
+                'UNIQUE_IDS': True,
-        'light': 'css/light.css',
+            },
+        },
+        'smarty': {
+            'smart_dashes': True,
+            'smart_quotes': False,
+            'smart_angled_quotes': False,
+            'smart_ellipses': True,
+        },
     }
 
-    TITLE_SUFFIX = 'incorporeal.org'
     MEDIA_DIR = 'media'
+
+    # customizations
+    PAGE_STYLES = {
+        'dark': '/static/css/dark.css',
+        'light': '/static/css/light.css',
+        'plain': '/static/css/plain.css',
+    }
+
+    DEFAULT_PAGE_STYLE = 'light'
+    TITLE_SUFFIX = 'example.com'
+    CONTACT_EMAIL = 'admin@example.com'
+
+    # specify FAVICON in your instance config.py to override the provided icon

incorporealcms/error_pages.py

@@ -0,0 +1,17 @@
+"""Error page views for 400, 404, etc."""
+from incorporealcms.lib import render
+
+
+def bad_request(error):
+    """Display 400 error messaging."""
+    return render('400.html'), 400
+
+
+def internal_server_error(error):
+    """Display 500 error messaging."""
+    return render('500.html'), 500
+
+
+def page_not_found(error):
+    """Display 404 error messaging."""
+    return render('404.html'), 404

incorporealcms/lib.py

@@ -0,0 +1,47 @@
+"""Miscellaneous helper functions and whatnot."""
+import logging
+
+import markdown
+from flask import current_app as app
+from flask import make_response, render_template, request
+
+logger = logging.getLogger(__name__)
+
+
+def get_meta_str(md, key):
+    """Provide the page's (parsed in Markup obj md) metadata for the specified key, or '' if unset."""
+    return " ".join(md.Meta.get(key)) if md.Meta.get(key) else ""
+
+
+def init_md():
+    """Initialize the Markdown parser.
+
+    This used to done at the app level in __init__, but extensions like footnotes apparently
+    assume the parser to only live for the length of parsing one document, and create double
+    footnote ref links if the one parser sees the same document multiple times.
+    """
+    # initialize markdown parser from config, but include
+    # extensions our app depends on, like the meta extension
+    return markdown.Markdown(extensions=app.config['MARKDOWN_EXTENSIONS'] + ['meta'],
+                             extension_configs=app.config['MARKDOWN_EXTENSION_CONFIGS'])
+
+
+def render(template_name_or_list, **context):
+    """Wrap Flask's render_template.
+
+    * Determine the proper site theme to use in the template and provide it.
+    """
+    page_styles = app.config['PAGE_STYLES']
+    selected_style = request.args.get('style', None)
+    if selected_style:
+        user_style = selected_style
+    else:
+        user_style = request.cookies.get('user-style')
+        logger.debug("user style cookie: %s", user_style)
+    context['user_style'] = page_styles.get(user_style, page_styles.get(app.config['DEFAULT_PAGE_STYLE']))
+    context['page_styles'] = page_styles
+
+    resp = make_response(render_template(template_name_or_list, **context))
+    if selected_style:
+        resp.set_cookie('user-style', selected_style)
+    return resp

incorporealcms/mdx/__init__.py

@@ -0,0 +1 @@
+"""Markdown extensions."""

incorporealcms/mdx/figures.py

@@ -0,0 +1,60 @@
+"""Create generic figures with captions."""
+import re
+from xml.etree.ElementTree import SubElement
+
+import markdown
+
+
+class FigureExtension(markdown.Extension):
+    """Wrap the markdown prepcoressor."""
+
+    def extendMarkdown(self, md):
+        """Add FigureBlockProcessor to the Markdown instance."""
+        md.parser.blockprocessors.register(FigureBlockProcessor(md.parser), 'figure', 100)
+
+
+class FigureBlockProcessor(markdown.blockprocessors.BlockProcessor):
+    """Process figures."""
+
+    # |> thing to put in the figure
+    # |: optional caption for the figure
+    # optional whatever else, like maybe an attr_list
+    figure_regex = re.compile(r'^[ ]{0,3}\|>[ ]{0,3}(?P<content>[^\n]*)')
+    caption_regex = re.compile(r'^[ ]{0,3}\|:[ ]{0,3}(?P<caption>[^\n]*)')
+
+    def test(self, parent, block):
+        """Determine if we should process this block."""
+        lines = block.split('\n')
+        return bool(self.figure_regex.search(lines[0]))
+
+    def run(self, parent, blocks):
+        """Replace the top block with HTML."""
+        block = blocks.pop(0)
+        lines = block.split('\n')
+
+        # consume line and create a figure
+        figure_match = self.figure_regex.search(lines[0])
+        lines.pop(0)
+        content = figure_match.group('content')
+        figure = SubElement(parent, 'figure')
+        figure.text = content
+        if lines:
+            if caption_match := self.caption_regex.search(lines[0]):
+                # consume line and add the caption as a child of the figure
+                lines.pop(0)
+                caption = caption_match.group('caption')
+                figcaption = SubElement(figure, 'figcaption')
+                figcaption.text = caption
+        if lines:
+            # other lines are mysteries, might be attr_list, so re-append
+            # make sure there's a child to hang the rest (which is maybe an attr_list?) off of
+            # this is probably a bad hack
+            if not len(list(figure)):
+                SubElement(figure, 'span')
+            rest = '\n'.join(lines)
+            figure[-1].tail = f'\n{rest}'
+
+
+def makeExtension(*args, **kwargs):
+    """Provide the extension to the markdown extension loader."""
+    return FigureExtension(*args, **kwargs)

incorporealcms/mdx/pydot.py

@@ -0,0 +1,54 @@
+"""Serve dot diagrams inline."""
+import base64
+import logging
+import re
+
+import markdown
+import pydot
+
+logger = logging.getLogger(__name__)
+
+
+class InlinePydot(markdown.Extension):
+    """Wrap the markdown prepcoressor."""
+
+    def extendMarkdown(self, md):
+        """Add InlinePydotPreprocessor to the Markdown instance."""
+        md.preprocessors.register(InlinePydotPreprocessor(md), 'dot_block', 100)
+
+
+class InlinePydotPreprocessor(markdown.preprocessors.Preprocessor):
+    """Identify dot codeblocks and run them through pydot."""
+
+    BLOCK_RE = re.compile(r'~~~{\s+pydot:(?P<filename>[^\s]+)\s+}\n(?P<content>.*?)~~~', re.DOTALL)
+
+    def run(self, lines):
+        """Match and generate diagrams from dot code blocks."""
+        text = '\n'.join(lines)
+        out = text
+        for block_match in self.BLOCK_RE.finditer(text):
+            filename = block_match.group(1)
+            dot_string = block_match.group(2)
+            logger.debug("matched markdown block: %s", dot_string)
+            logger.debug("match start/end: %s/%s", block_match.start(), block_match.end())
+
+            # use pydot to turn the text into pydot
+            graphs = pydot.graph_from_dot_data(dot_string)
+            if not graphs:
+                logger.debug("some kind of issue with parsed 'dot' %s", dot_string)
+                raise ValueError("error parsing dot text!")
+
+            # encode the image and provide as an inline image in markdown
+            encoded_image = base64.b64encode(graphs[0].create_png()).decode('ascii')
+            data_path = f'data:image/png;base64,{encoded_image}'
+            inline_image = f'![{filename}]({data_path})'
+
+            # replace the image in the output markdown
+            out = out.replace(block_match.group(0), inline_image)
+
+        return out.split('\n')
+
+
+def makeExtension(*args, **kwargs):
+    """Provide the extension to the markdown extension loader."""
+    return InlinePydot(*args, **kwargs)

incorporealcms/pages.py

@@ -2,11 +2,14 @@
 import datetime
 import logging
 import os
+import re
 
 from flask import Blueprint, Markup, abort
 from flask import current_app as app
-from flask import make_response, redirect, render_template, request
+from flask import redirect, request, send_from_directory
-from tzlocal import get_localzone
+from werkzeug.security import safe_join
+
+from incorporealcms.lib import get_meta_str, init_md, render
 
 logger = logging.getLogger(__name__)
 
@@ -17,92 +20,184 @@ bp = Blueprint('pages', __name__, url_prefix='/')
 @bp.route('/<path:path>')
 def display_page(path):
     """Get the file contents of the requested path and render the file."""
-    if is_file_path_actually_dir_path(path):
-        return redirect(f'{path}/', code=301)
-
-    resolved_path = resolve_page_file(path)
-    logger.debug("received request for path '%s', resolved to '%s'", path, resolved_path)
     try:
-        with app.open_instance_resource(resolved_path, 'r') as entry_file:
+        resolved_path, render_type = request_path_to_instance_resource_path(path)
-            logger.debug("file '%s' found", resolved_path)
+        logger.debug("received request for path '%s', resolved to '%s', type '%s'",
-            parent_navs = generate_parent_navs(path)
+                     path, resolved_path, render_type)
-            mtime = datetime.datetime.fromtimestamp(os.path.getmtime(entry_file.name), get_localzone())
+    except PermissionError:
-            entry = entry_file.read()
+        abort(400)
+    except IsADirectoryError:
+        return redirect(f'/{path}/', code=301)
     except FileNotFoundError:
-        logger.warning("requested path '%s' (resolved path '%s') not found!", path, resolved_path)
         abort(404)
+
+    if render_type == 'file':
+        return send_from_directory(app.instance_path, resolved_path)
+    elif render_type == 'symlink':
+        logger.debug("attempting to redirect path '%s' to reverse of resource '%s'", path, resolved_path)
+        redirect_path = f'/{instance_resource_path_to_request_path(resolved_path)}'
+        logger.debug("redirect path: '%s'", redirect_path)
+        return redirect(redirect_path, code=301)
+    elif render_type == 'markdown':
+        logger.debug("treating path '%s' as markdown '%s'", path, resolved_path)
+        return handle_markdown_file_path(resolved_path)
     else:
-        content = Markup(app.config['md'].convert(entry))
+        logger.exception("unsupported render_type '%s'!?", render_type)
-        logger.debug("file metadata: %s", app.config['md'].Meta)
+        abort(500)
-        title = " ".join(app.config['md'].Meta.get('title')) if app.config['md'].Meta.get('title') else ""
-        return render('base.html', title=title, content=content, navs=parent_navs,
-                      mtime=mtime.strftime('%Y-%m-%d %H:%M:%S %Z'))
 
 
-def render(template_name_or_list, **context):
+def handle_markdown_file_path(resolved_path):
-    """Wrap Flask's render_template.
+    """Given a location on disk, attempt to open it and render the markdown within."""
-
+    try:
-    * Determine the proper site theme to use in the template and provide it.
+        logger.debug("opening resolved path '%s'", resolved_path)
-    """
+        with app.open_instance_resource(resolved_path, 'r') as entry_file:
-    selected_style = request.args.get('style', None)
+            mtime = datetime.datetime.fromtimestamp(os.path.getmtime(entry_file.name), tz=datetime.timezone.utc)
-    if selected_style:
+            entry = entry_file.read()
-        user_style = selected_style
+        logger.debug("resolved path '%s' read", resolved_path)
+    except OSError:
+        logger.exception("resolved path '%s' could not be opened!", resolved_path)
+        abort(500)
     else:
-        user_style = request.cookies.get('user-style')
+        try:
-        logger.debug("user style cookie: %s", user_style)
+            md = init_md()
-    context['user_style'] = app.config['PAGE_STYLES'].get(user_style, app.config['PAGE_STYLES']['DEFAULT'])
+            content = Markup(md.convert(entry))
+        except ValueError:
+            logger.exception("error parsing/rendering markdown!")
+            abort(500)
+        except TypeError:
+            logger.exception("error loading/rendering markdown!")
+            abort(500)
 
-    resp = make_response(render_template(template_name_or_list, **context))
+        logger.debug("file metadata: %s", md.Meta)
-    if selected_style:
+
-        resp.set_cookie('user-style', selected_style)
+        parent_navs = generate_parent_navs(resolved_path)
-    return resp
+
+        page_name = (get_meta_str(md, 'title') if md.Meta.get('title') else
+                     f'/{instance_resource_path_to_request_path(resolved_path)}')
+        page_title = f'{page_name} - {app.config["TITLE_SUFFIX"]}' if page_name else app.config['TITLE_SUFFIX']
+        logger.debug("title (potentially derived): %s", page_title)
+
+        extra_footer = get_meta_str(md, 'footer') if md.Meta.get('footer') else None
+
+        template = get_meta_str(md, 'template') if md.Meta.get('template') else 'base.html'
+
+        # check if this has a HTTP redirect
+        redirect_url = get_meta_str(md, 'redirect') if md.Meta.get('redirect') else None
+        if redirect_url:
+            logger.debug("redirecting via meta tag to '%s'", redirect_url)
+            return redirect(redirect_url, code=301)
+
+        return render(template, title=page_title, description=get_meta_str(md, 'description'),
+                      image=get_meta_str(md, 'image'), base_url=request.base_url, content=content,
+                      navs=parent_navs, mtime=mtime.strftime('%Y-%m-%d %H:%M:%S %Z'),
+                      extra_footer=extra_footer)
 
 
-def resolve_page_file(path):
+def request_path_to_instance_resource_path(path):
-    """Manipulate the request path to find appropriate page file.
+    """Turn a request URL path to the full page path.
 
-    * convert dir requests to index files
+    flask.Flask.open_instance_resource will open a file like /etc/hosts if you tell it to,
-
+    which sucks, so we do a lot of work here to make sure we have a valid request to
-    Worth noting, Flask already does stuff like convert '/foo/../../../bar' to
+    something inside the pages dir.
-    '/bar', so we don't need to take care around file access here.
     """
-    if path.endswith('/'):
+    # check if the path is allowed
-        path = f'{path}index'
+    base_dir = os.path.realpath(f'{app.instance_path}/pages/')
-    path = f'pages/{path}.md'
+    safe_path = safe_join(base_dir, path)
-    return path
+    # bail if the requested real path isn't inside the base directory
+    if not safe_path:
+        logger.warning("client tried to request a path '%s' outside of the base_dir!", path)
+        raise PermissionError
+
+    verbatim_path = os.path.abspath(safe_path)
+    resolved_path = os.path.realpath(verbatim_path)
+    logger.debug("base_dir '%s', constructed resolved_path '%s' for path '%s'", base_dir, resolved_path, path)
+
+    # see if we have a real file or if we should infer markdown rendering
+    if os.path.exists(resolved_path):
+        # if this is a file-like request but actually a directory, redirect the user
+        if os.path.isdir(resolved_path) and not path.endswith('/'):
+            logger.info("client requested a path '%s' that is actually a directory", path)
+            raise IsADirectoryError
+
+        # if the requested path contains a symlink, redirect the user
+        if verbatim_path != resolved_path:
+            logger.info("client requested a path '%s' that is actually a symlink to file '%s'", path, resolved_path)
+            return resolved_path.replace(f'{app.instance_path}{os.path.sep}', ''), 'symlink'
+
+        # derive the proper markdown or actual file depending on if this is a dir or file
+        if os.path.isdir(resolved_path):
+            resolved_path = os.path.join(resolved_path, 'index.md')
+            return resolved_path.replace(f'{app.instance_path}{os.path.sep}', ''), 'markdown'
+
+        logger.info("final DIRECT path = '%s' for request '%s'", resolved_path, path)
+        return resolved_path.replace(f'{app.instance_path}{os.path.sep}', ''), 'file'
+
+    # if we're here, this isn't direct file access, so try markdown inference
+    verbatim_path = f'{safe_path}.md'
+    resolved_path = os.path.realpath(verbatim_path)
+
+    # does the final file actually exist?
+    if not os.path.exists(resolved_path):
+        logger.warning("requested final path '%s' does not exist!", resolved_path)
+        raise FileNotFoundError
+
+    # check for symlinks
+    if verbatim_path != resolved_path:
+        logger.info("client requested a path '%s' that is actually a symlink to file '%s'", path, resolved_path)
+        return resolved_path.replace(f'{app.instance_path}{os.path.sep}', ''), 'symlink'
+
+    logger.info("final path = '%s' for request '%s'", resolved_path, path)
+    # we checked that the file exists via absolute path, but now we need to give the path relative to instance dir
+    return resolved_path.replace(f'{app.instance_path}{os.path.sep}', ''), 'markdown'
 
 
-def is_file_path_actually_dir_path(path):
+def instance_resource_path_to_request_path(path):
-    """Check if requested path which looks like a file is actually a directory.
+    """Reverse a (presumed to exist) disk path to the canonical path that would show up in a Flask route.
 
-    If, for example, /foo used to be a file (foo.md) which later became a directory,
+    This does not include the leading /, so aside from the root index case, this should be
-    foo/, this returns True. Useful for when I make my structure more complicated
+    bidirectional.
-    than it originally was, or if users are just weird.
     """
-    if not path.endswith('/'):
+    return re.sub(r'^pages/', '', re.sub(r'.md$', '', re.sub(r'index.md$', '', path)))
-        logger.debug("requested path '%s' looks like a file", path)
-        if os.path.isdir(f'{app.instance_path}/pages/{path}'):
-            logger.debug("...and it's actually a dir")
-            return True
-
-    return False
 
 
 def generate_parent_navs(path):
-    """Create a series of paths/links to navigate up from the given path."""
+    """Create a series of paths/links to navigate up from the given resource path."""
-    # derive additional path/location stuff based on path
+    if path == 'pages/index.md':
-    resolved_path = resolve_page_file(path)
+        # bail and return the title suffix (generally the domain name) as a terminal case
-    parent_dir = os.path.dirname(resolved_path)
-    parent_path = '/'.join(path[:-1].split('/')[:-1]) + '/'
-
-    logger.debug("path: '%s'; parent path: '%s'; resolved path: '%s'; parent dir: '%s'",
-                 path, parent_path, resolved_path, parent_dir)
-
-    if path in ('index', '/'):
         return [(app.config['TITLE_SUFFIX'], '/')]
     else:
-        with app.open_instance_resource(resolved_path, 'r') as entry_file:
+        if path.endswith('index.md'):
+            # index case: one dirname for foo/bar/index.md -> foo/bar, one for foo/bar -> foo
+            parent_resource_dir = os.path.dirname(os.path.dirname(path))
+        else:
+            # usual case: foo/buh.md -> foo
+            parent_resource_dir = os.path.dirname(path)
+
+        # generate the request path (i.e. what the link will be) for this path, and
+        # also the resource path of this parent (which is always a dir, so always index.md)
+        request_path = f'/{instance_resource_path_to_request_path(path)}'
+        parent_resource_path = os.path.join(parent_resource_dir, 'index.md')
+
+        logger.debug("resource path: '%s'; request path: '%s'; parent resource path: '%s'", path,
+                     request_path, parent_resource_path)
+
+        # for issues regarding parser reuse (see lib.init_md) we reinitialize the parser here
+        md = init_md()
+
+        # read the resource
+        try:
+            with app.open_instance_resource(path, 'r') as entry_file:
                 entry = entry_file.read()
-        _ = Markup(app.config['md'].convert(entry))
+            _ = Markup(md.convert(entry))
-        page_name = " ".join(app.config['md'].Meta.get('title')) if app.config['md'].Meta.get('title') else f'/{path}'
+            page_name = (" ".join(md.Meta.get('title')) if md.Meta.get('title')
-        return generate_parent_navs(parent_path) + [(page_name, f'/{path}')]
+                         else request_path_to_breadcrumb_display(request_path))
+            return generate_parent_navs(parent_resource_path) + [(page_name, request_path)]
+        except FileNotFoundError:
+            return generate_parent_navs(parent_resource_path) + [(request_path, request_path)]
+
+
+def request_path_to_breadcrumb_display(path):
+    """Given a request path, e.g. "/foo/bar/baz/", turn it into breadcrumby text "baz"."""
+    undired = path.rstrip('/')
+    leaf = undired[undired.rfind('/'):]
+    return leaf.strip('/')

incorporealcms/static.py

@@ -0,0 +1,14 @@
+"""Serve static files from the instance directory."""
+import os
+
+from flask import Blueprint
+from flask import current_app as app
+from flask import send_from_directory
+
+bp = Blueprint('static', __name__, url_prefix='/custom-static')
+
+
+@bp.route('/<path:name>')
+def serve_instance_static_file(name):
+    """Serve a static file from the instance directory, used for customization."""
+    return send_from_directory(os.path.join(app.instance_path, 'custom-static'), name)

incorporealcms/static/css/base.css

@@ -0,0 +1,171 @@
+html {
+    font-family: sans-serif;
+    padding: 0;
+}
+
+body {
+    margin: 0;
+    text-size-adjust: 100%;
+    -ms-text-size-adjust: 100%;
+    -moz-text-size-adjust: 100%;
+    -webkit-text-size-adjust: 100%;
+}
+
+.site-wrap-normal-width {
+    max-width: 65pc;
+}
+
+.site-wrap-double-width {
+    max-width: 130pc;
+}
+
+.site-wrap {
+    min-height: 100vh;
+    margin: 0;
+    margin-left: auto;
+    margin-right: auto;
+}
+
+a {
+    font-weight: bold;
+    text-decoration-line: underline;
+    text-decoration-thickness: 1px;
+}
+
+div.header {
+    display: flex;
+    justify-content: space-between;
+    font-size: 0.8em;
+    padding: 1rem 1rem;
+    padding-bottom: 0;
+}
+
+div.content {
+    font-size: 11pt;
+    padding: 0 1rem;
+    line-height: 1.6em;
+}
+
+div.content p {
+    margin: 1.25em 0;
+}
+
+sup, sub {
+	vertical-align: baseline;
+	position: relative;
+	top: -0.4em;
+}
+sub {
+	top: 0.4em;
+}
+
+footer {
+    clear: both;
+    display: block;
+    font-size: 75%;
+    color: #999;
+    padding: 0 1em;
+    padding-bottom: 16px;
+    margin-top: 30px;
+}
+
+.extra-footer {
+    margin-bottom: 5px;
+}
+
+table {
+    border-collapse: collapse;
+}
+
+table, th, td {
+    padding: 5px;
+    border: 1px solid #ccc;
+    margin-bottom: 15px;
+}
+
+blockquote {
+    background-color: rgba(120, 120, 120, 0.3);
+    padding: 1px 10px;
+}
+
+.splash {
+    margin-top: 1em;
+    margin-bottom: 1em;
+    text-align: center;
+}
+
+img {
+    max-width: 100%;
+}
+
+.img-25 {
+    max-width: 25% !important;
+}
+
+.img-50 {
+    max-width: 50% !important;
+}
+
+.img-75 {
+    max-width: 75% !important;
+}
+
+.img-center {
+    display: block;
+    margin-left: auto;
+    margin-right: auto;
+}
+
+.img-left {
+    float: left;
+    margin-right: 1em;
+}
+
+.img-right {
+    float: right;
+    margin-left: 1em;
+}
+
+figure {
+    padding: 5px;
+    margin: 0;
+    margin-bottom: 5px;
+    text-align: center;
+    max-width: 30%;
+    min-width: 10em;
+    display: inline-block;
+}
+
+figure.right {
+    float: right;
+    margin-left: 10px;
+    display: block;
+}
+
+figure.left {
+    float: left;
+    margin-right: 10px;
+    display: block;
+}
+
+figure img {
+    max-width: 100%;
+    height: auto;
+}
+
+figcaption {
+    text-align: center;
+    font-size: 0.9em;
+}
+
+.footnote {
+    font-size: 0.8em;
+}
+
+.footnote p {
+    margin: 0;
+}
+
+.footnote-ref:link, .footnote-ref:visited, .footnote-ref:hover, .footnote-ref:active {
+    font-weight: normal;
+}

incorporealcms/static/css/dark.css

@@ -1,41 +1,37 @@
+/* common styling via the base.css, used in light and dark */
+@import '/static/css/base.css';
+
 html {
-    color: #CCC;
+    color: #DDD;
 }
 
 body {
-    background: #444;
+    background: #111;
-}
-
-.site-wrap {
-    background: black;
-
-    border: 1px solid #222;
-    border-top: none;
-    border-bottom: none;
 }
 
 h1, h2, h3, h4, h5, h6 {
     color: #B31D15;
 }
 
-a:link, a:visited {
+p a, ul a, ol a {
-    color: #BBB;
+    color: #DDD;
-    border-bottom: 1px dotted #CCC;
 }
 
-a:hover, a:active {
+footer a {
-    color: #B14640;
+    color: #999;
-    border-bottom: 1px dotted #CCC;
 }
 
-section.nav, section.styles {
+p a:hover, ul a:hover, ol a:hover, footer a:hover {
-    color: #BBB;
+    color: #B31D15;
-    background: #222;
-    border-bottom: 1px solid #222;
 }
 
-section.nav a, section.styles a {
+div.site-wrap {
-    color: #BBB;
+    background: black;
+}
+
+div.header, div.header a {
+    color: #555;
+    text-decoration: none;
 }
 
 table, th, td {
@@ -50,3 +46,12 @@ blockquote {
     background-color: rgba(120, 120, 120, 0.3);
     border: 1px solid #222;
 }
+
+figure {
+    background-color: rgba(255, 255, 255, 0.1);
+    border: 1px solid #333;
+}
+
+figcaption {
+    color: #AAAAAA;
+}

incorporealcms/static/css/light.css

@@ -1,52 +1,57 @@
+/* common styling via the base.css, used in light and dark */
+@import '/static/css/base.css';
+
 html {
     color: #222;
 }
 
 body {
-    background: #888;
+    background: #EEE;
-}
-
-.site-wrap {
-    background: white;
-
-    border: 1px solid #ddd;
-    border-top: none;
-    border-bottom: none;
 }
 
 h1, h2, h3, h4, h5, h6 {
     color: #811610;
 }
 
-a:link, a:visited {
+p a, ul a, ol a {
     color: #222;
-    border-bottom: 1px dotted #222;
 }
 
-a:hover, a:active {
+footer a {
+    color: #999;
+}
+
+p a:hover, ul a:hover, ol a:hover, footer a:hover {
     color: #811610;
-    border-bottom: 1px dotted #222;
 }
 
-section.nav, section.styles {
+div.site-wrap {
-    color: #666;
+    background: white;
-    background: #EEE;
-    border-bottom: 1px solid #CCC;
 }
 
-section.nav a, section.styles a {
+div.header, div.header a {
-    color: #666;
+    color: #AAA;
+    text-decoration: none;
 }
 
 table, th, td {
-    border: 1px solid #ccc;
+    border: 1px solid #CCC;
 }
 
 th {
-    background: #eee;
+    background: #CCC;
 }
 
 blockquote {
     background-color: rgba(120, 120, 120, 0.1);
+    border: 1px solid #DDD;
+}
+
+figure {
+    background-color: rgba(0, 0, 0, 0.1);
     border: 1px solid #CCC;
 }
+
+figcaption {
+    color: #666666;
+}

incorporealcms/static/css/plain.css

@@ -0,0 +1,33 @@
+/* specify almost no styling, just fix some image and nav rendering */
+div.header {
+    display: flex;
+    justify-content: space-between;
+}
+
+.img-25 {
+    max-width: 25% !important;
+}
+
+.img-50 {
+    max-width: 50% !important;
+}
+
+.img-75 {
+    max-width: 75% !important;
+}
+
+.img-center {
+    display: block;
+    margin-left: auto;
+    margin-right: auto;
+}
+
+.img-left {
+    float: left;
+    margin-right: 1em;
+}
+
+.img-right {
+    float: right;
+    margin-left: 1em;
+}

incorporealcms/static/css/style.css

@@ -1,106 +0,0 @@
-html {
-    font-family: sans-serif;
-    padding: 0;
-}
-
-body {
-
-    margin: 0;
-}
-
-.site-wrap {
-    max-width: 70pc;
-    min-height: 100vh;
-    margin: 0;
-    margin-left: auto;
-    margin-right: auto;
-}
-
-h1 {
-    font-size: 2em;
-}
-
-h2 {
-    font-size: 1.5em;
-}
-
-h3 {
-    font-size: 1.25em;
-}
-
-h4 {
-    font-size: 1.17em;
-}
-
-h5 {
-    font-size: 1em;
-}
-
-h6 {
-    font-size: .83em;
-}
-
-a:link {
-    font-weight: bold;
-    text-decoration: none;
-}
-
-a:visited {
-    font-weight: bold;
-    text-decoration: none;
-}
-
-a:hover {
-    font-weight: bold;
-    text-decoration: none;
-}
-
-a:active {
-    font-weight: bold;
-    text-decoration: none;
-}
-
-section.nav, section.styles {
-    font-size: 0.75em;
-    border-bottom: 1px solid #444;
-    padding: 0.25em 0.5em;
-}
-
-section.nav a, section.styles a {
-    border-bottom: none;
-}
-
-section.styles {
-    float: right;
-}
-
-section.content {
-    font-size: 11pt;
-
-    padding: 0 1em;
-    line-height: 1.5em;
-}
-
-footer {
-    display: block;
-    font-size: 75%;
-    color: #999;
-    padding: 0 1em;
-    padding-bottom: 16px;
-    margin-top: 15px;
-}
-
-table {
-    border-collapse: collapse;
-}
-
-table, th, td {
-    padding: 5px;
-    border: 1px solid #ccc;
-    margin-bottom: 15px;
-}
-
-blockquote {
-    background-color: rgba(120, 120, 120, 0.3);
-    padding: 1px 10px;
-}

incorporealcms/templates/400.html

@@ -0,0 +1,14 @@
+{% extends "base.html" %}
+
+{% block header %}
+<div class="header">
+    <a href="/">{{ config.TITLE_SUFFIX }}</a>
+</div>
+{% endblock %}
+
+{% block body %}
+<div class="content">
+    <h1>BAD REQUEST</h1>
+    <p>You're doing something you're not supposed to. Stop it?</p>
+</div>
+{% endblock %}

incorporealcms/templates/404.html

@@ -0,0 +1,19 @@
+{% extends "base.html" %}
+
+{% block header %}
+<div class="header">
+    <a href="/">{{ config.TITLE_SUFFIX }}</a>
+</div>
+{% endblock %}
+
+{% block body %}
+<div class="content">
+    <h1>NOT FOUND</h1>
+    <p>Sorry, <b><tt>{{ request.path }}</tt></b> does not seem to exist, at least not anymore.</p>
+    <p>It's possible you followed a dead link on this site, in which case I would appreciate it if you could email me at
+    {{ config.CONTACT_EMAIL }} and I can take a look. I make an effort to symlink old content to its new location,
+    so old links and URLs should, generally speaking, work.</p>
+    <p>Otherwise, I suggest you go <a href="/">to the index</a> and navigate your way (hopefully) to what
+    you're looking for.</p>
+</div>
+{% endblock %}

incorporealcms/templates/500.html

@@ -0,0 +1,14 @@
+{% extends "base.html" %}
+
+{% block header %}
+<div class="header">
+    <a href="/">{{ config.TITLE_SUFFIX }}</a>
+</div>
+{% endblock %}
+
+{% block body %}
+<div class="content">
+    <h1>INTERNAL SERVER ERROR</h1>
+    <p>Something bad happened! Please email me at {{ config.CONTACT_EMAIL }} and tell me what happened.</p>
+</div>
+{% endblock %}

incorporealcms/templates/base-wide.html

@@ -0,0 +1,2 @@
+{% extends "base.html" %}
+{% block site_class %}class="site-wrap site-wrap-double-width"{% endblock %}

incorporealcms/templates/base.html

@@ -1,23 +1,39 @@
 <!doctype html>
-<title>{{ title }}{% if title %} - {% endif %}{{ config.TITLE_SUFFIX }}</title>
+<html lang="en">
-<link rel="stylesheet" href="{{ url_for('static', filename='css/style.css') }}">
+<title>{{ title }}</title>
-<link rel="stylesheet" href="{{ url_for('static', filename=user_style) }}">
+{% if title %}<meta property="og:title" content="{{ title }}">{% endif %}
-<link rel="icon" href="{{ url_for('static', filename='img/favicon.png') }}">
+{% if description %}<meta property="og:description" content="{{ description }}">{% endif %}
-<section class="site-wrap">
+{% if image %}<meta property="og:image" content="{{ image }}">{% endif %}
-    <section class="styles">
+<meta property="og:url" content="{{ base_url }}">
-        <a href="?style=dark">[dark]</a>
+<meta name="twitter:card" content="summary_large_image">
-        <a href="?style=light">[light]</a>
+<meta name="viewport" content="width=device-width, initial-scale=1">
-    </section>
+<link rel="stylesheet" href="{{ user_style }}">
-    <section class="nav">
+<link rel="icon" href="{% if config.FAVICON %}{{ config.FAVICON }}{% else %}{{ url_for('static', filename='img/favicon.png') }}{% endif %}">
+
+<div {% block site_class %}class="site-wrap site-wrap-normal-width"{% endblock %}>
+    {% block header %}
+    <div class="header">
+        <div class="nav">
             {% for nav in navs %}
             <a href="{{ nav.1 }}">{{ nav.0 }}</a>
             {% if not loop.last %} &raquo; {% endif %}
             {% endfor %}
-    </section>
+        </div>
-    <section class="content">
+        <div class="styles">
+            {% for style in page_styles %}
+            <a href="?style={{ style }}">[{{ style }}]</a>
+            {% endfor %}
+        </div>
+    </div>
+    {% endblock %}
+    {% block body %}
+    <div class="content">
         {{ content }}
-    </section>
+    </div>
     <footer>
-        <i>Last modified: {{ mtime }}</i>
+        {% if extra_footer %}<div class="extra-footer"><i>{{ extra_footer|safe }}</i></div>{% endif %}
+        <div class="footer"><i>Last modified: {{ mtime }}</i></div>
     </footer>
-</section>
+    {% endblock %}
+</div>
+</html>

requirements/requirements-dev.in

@@ -1,6 +1,7 @@
 -r requirements.in
 
 # testing runner, test reporting, packages used during testing (e.g. requests-mock), etc.
+pydot
 pytest
 pytest-cov
 
@@ -16,9 +17,10 @@ flake8-fixme
 flake8-isort
 flake8-logging-format
 flake8-mutable
+safety                      # check requirements file for issues
 
 # maintenance utilities and tox
 pip-tools                   # pip-compile
-tox                         # CI stuff
+tox<4                       # CI stuff, pinned for now to avoid packaging conflict w/safety
 tox-wheel                   # build wheels in tox
 versioneer                  # automatic version numbering

requirements/requirements-dev.txt

@@ -1,64 +1,171 @@
 #
-# This file is autogenerated by pip-compile
+# This file is autogenerated by pip-compile with Python 3.10
-# To update, run:
+# by the following command:
 #
 #    pip-compile --output-file=requirements/requirements-dev.txt requirements/requirements-dev.in
 #
-appdirs==1.4.4            # via virtualenv
+attrs==22.2.0
-attrs==20.2.0             # via pytest
+    # via pytest
-bandit==1.6.2             # via -r requirements/requirements-dev.in
+bandit==1.7.4
-click==7.1.2              # via flask, pip-tools
+    # via -r requirements/requirements-dev.in
-coverage==5.3             # via pytest-cov
+build==0.9.0
-distlib==0.3.1            # via virtualenv
+    # via pip-tools
-dlint==0.10.3             # via -r requirements/requirements-dev.in
+certifi==2022.12.7
-filelock==3.0.12          # via tox, virtualenv
+    # via requests
-flake8-blind-except==0.1.1  # via -r requirements/requirements-dev.in
+charset-normalizer==2.1.1
-flake8-builtins==1.5.3    # via -r requirements/requirements-dev.in
+    # via requests
-flake8-docstrings==1.5.0  # via -r requirements/requirements-dev.in
+click==8.1.3
-flake8-executable==2.0.4  # via -r requirements/requirements-dev.in
+    # via
-flake8-fixme==1.1.1       # via -r requirements/requirements-dev.in
+    #   flask
-flake8-isort==4.0.0       # via -r requirements/requirements-dev.in
+    #   pip-tools
-flake8-logging-format==0.6.0  # via -r requirements/requirements-dev.in
+    #   safety
-flake8-mutable==1.2.0     # via -r requirements/requirements-dev.in
+coverage[toml]==7.0.1
-flake8==3.8.4             # via -r requirements/requirements-dev.in, dlint, flake8-builtins, flake8-docstrings, flake8-executable, flake8-isort, flake8-mutable
+    # via pytest-cov
-flask==1.1.2              # via -r requirements/requirements.in
+distlib==0.3.6
-gitdb==4.0.5              # via gitpython
+    # via virtualenv
-gitpython==3.1.9          # via bandit
+dlint==0.13.0
-importlib-metadata==2.0.0  # via flake8, markdown, pluggy, pytest, stevedore, tox, virtualenv
+    # via -r requirements/requirements-dev.in
-iniconfig==1.1.1          # via pytest
+dparse==0.6.2
-isort==5.6.4              # via flake8-isort
+    # via safety
-itsdangerous==1.1.0       # via flask
+exceptiongroup==1.1.0
-jinja2==2.11.2            # via flask
+    # via pytest
-markdown==3.3.2           # via -r requirements/requirements.in
+filelock==3.9.0
-markupsafe==1.1.1         # via jinja2
+    # via
-mccabe==0.6.1             # via flake8
+    #   tox
-packaging==20.4           # via pytest, tox
+    #   virtualenv
-pbr==5.5.1                # via stevedore
+flake8==5.0.4
-pip-tools==5.3.1          # via -r requirements/requirements-dev.in
+    # via
-pluggy==0.13.1            # via pytest, tox
+    #   -r requirements/requirements-dev.in
-py==1.9.0                 # via pytest, tox
+    #   dlint
-pycodestyle==2.6.0        # via flake8
+    #   flake8-builtins
-pydocstyle==5.1.1         # via flake8-docstrings
+    #   flake8-docstrings
-pyflakes==2.2.0           # via flake8
+    #   flake8-executable
-pyparsing==2.4.7          # via packaging
+    #   flake8-isort
-pytest-cov==2.10.1        # via -r requirements/requirements-dev.in
+    #   flake8-mutable
-pytest==6.1.1             # via -r requirements/requirements-dev.in, pytest-cov
+flake8-blind-except==0.2.1
-pytz==2020.1              # via tzlocal
+    # via -r requirements/requirements-dev.in
-pyyaml==5.3.1             # via bandit
+flake8-builtins==2.1.0
-six==1.15.0               # via bandit, packaging, pip-tools, tox, virtualenv
+    # via -r requirements/requirements-dev.in
-smmap==3.0.4              # via gitdb
+flake8-docstrings==1.6.0
-snowballstemmer==2.0.0    # via pydocstyle
+    # via -r requirements/requirements-dev.in
-stevedore==3.2.2          # via bandit
+flake8-executable==2.1.2
-testfixtures==6.15.0      # via flake8-isort
+    # via -r requirements/requirements-dev.in
-toml==0.10.1              # via pytest, tox
+flake8-fixme==1.1.1
-tox-wheel==0.5.0          # via -r requirements/requirements-dev.in
+    # via -r requirements/requirements-dev.in
-tox==3.20.1               # via -r requirements/requirements-dev.in, tox-wheel
+flake8-isort==6.0.0
-tzlocal==2.1              # via -r requirements/requirements.in
+    # via -r requirements/requirements-dev.in
-versioneer==0.18          # via -r requirements/requirements-dev.in
+flake8-logging-format==0.9.0
-virtualenv==20.0.35       # via tox
+    # via -r requirements/requirements-dev.in
-werkzeug==1.0.1           # via flask
+flake8-mutable==1.2.0
-wheel==0.35.1             # via tox-wheel
+    # via -r requirements/requirements-dev.in
-zipp==3.3.1               # via importlib-metadata
+flask==2.2.2
+    # via -r requirements/requirements.in
+gitdb==4.0.10
+    # via gitpython
+gitpython==3.1.30
+    # via bandit
+idna==3.4
+    # via requests
+iniconfig==1.1.1
+    # via pytest
+isort==5.11.4
+    # via flake8-isort
+itsdangerous==2.1.2
+    # via flask
+jinja2==3.1.2
+    # via flask
+markdown==3.4.1
+    # via -r requirements/requirements.in
+markupsafe==2.1.1
+    # via
+    #   jinja2
+    #   werkzeug
+mccabe==0.7.0
+    # via flake8
+packaging==21.3
+    # via
+    #   build
+    #   dparse
+    #   pytest
+    #   safety
+    #   tox
+pbr==5.11.0
+    # via stevedore
+pep517==0.13.0
+    # via build
+pip-tools==6.12.1
+    # via -r requirements/requirements-dev.in
+platformdirs==2.6.2
+    # via virtualenv
+pluggy==1.0.0
+    # via
+    #   pytest
+    #   tox
+py==1.11.0
+    # via tox
+pycodestyle==2.9.1
+    # via flake8
+pydocstyle==6.1.1
+    # via flake8-docstrings
+pydot==1.4.2
+    # via -r requirements/requirements-dev.in
+pyflakes==2.5.0
+    # via flake8
+pyparsing==3.0.9
+    # via
+    #   packaging
+    #   pydot
+pytest==7.2.0
+    # via
+    #   -r requirements/requirements-dev.in
+    #   pytest-cov
+pytest-cov==4.0.0
+    # via -r requirements/requirements-dev.in
+pyyaml==6.0
+    # via bandit
+requests==2.28.1
+    # via safety
+ruamel-yaml==0.17.21
+    # via safety
+ruamel-yaml-clib==0.2.7
+    # via ruamel-yaml
+safety==2.3.5
+    # via -r requirements/requirements-dev.in
+six==1.16.0
+    # via tox
+smmap==5.0.0
+    # via gitdb
+snowballstemmer==2.2.0
+    # via pydocstyle
+stevedore==4.1.1
+    # via bandit
+toml==0.10.2
+    # via dparse
+tomli==2.0.1
+    # via
+    #   build
+    #   coverage
+    #   pep517
+    #   pytest
+    #   tox
+tox==3.28.0
+    # via
+    #   -r requirements/requirements-dev.in
+    #   tox-wheel
+tox-wheel==1.0.0
+    # via -r requirements/requirements-dev.in
+urllib3==1.26.13
+    # via requests
+versioneer==0.28
+    # via -r requirements/requirements-dev.in
+virtualenv==20.17.1
+    # via tox
+werkzeug==2.2.2
+    # via flask
+wheel==0.38.4
+    # via
+    #   pip-tools
+    #   tox-wheel
 
 # The following packages are considered to be unsafe in a requirements file:
 # pip

requirements/requirements.in

@@ -1,3 +1,2 @@
 Flask                       # general purpose web service and web server stuff
 Markdown                    # markdown rendering in templates
-tzlocal                     # identifying system's local timezone

requirements/requirements.txt

@@ -1,17 +1,22 @@
 #
-# This file is autogenerated by pip-compile
+# This file is autogenerated by pip-compile with python 3.10
 # To update, run:
 #
 #    pip-compile --output-file=requirements/requirements.txt requirements/requirements.in
 #
-click==7.1.2              # via flask
+click==8.1.3
-flask==1.1.2              # via -r requirements/requirements.in
+    # via flask
-importlib-metadata==2.0.0  # via markdown
+flask==2.2.2
-itsdangerous==1.1.0       # via flask
+    # via -r requirements/requirements.in
-jinja2==2.11.2            # via flask
+itsdangerous==2.1.2
-markdown==3.3.2           # via -r requirements/requirements.in
+    # via flask
-markupsafe==1.1.1         # via jinja2
+jinja2==3.1.2
-pytz==2020.1              # via tzlocal
+    # via flask
-tzlocal==2.1              # via -r requirements/requirements.in
+markdown==3.4.1
-werkzeug==1.0.1           # via flask
+    # via -r requirements/requirements.in
-zipp==3.3.1               # via importlib-metadata
+markupsafe==2.1.1
+    # via
+    #   jinja2
+    #   werkzeug
+werkzeug==2.2.2
+    # via flask

setup.py

@@ -18,13 +18,19 @@ setup(
     name='incorporeal-cms',
     description='Flask project for running https://suou.net (and eventually others).',
     url='https://git.incorporeal.org/bss/incorporeal-cms',
-    license='GPL3',
     author='Brian S. Stephan',
     author_email='bss@incorporeal.org',
+    license='AGPLv3+',
+    classifiers=[
+        'License :: OSI Approved :: GNU Affero General Public License v3 or later (AGPLv3+)',
+    ],
     version=versioneer.get_version(),
     cmdclass=versioneer.get_cmdclass(),
     packages=find_packages(),
     include_package_data=True,
     zip_safe=False,
     install_requires=extract_requires(),
+    extras_require={
+        'dot': ['pydot'],
+    },
 )

tests/functional_markdown_tests.py

@@ -0,0 +1,63 @@
+"""Test graphviz functionality."""
+import os
+
+from incorporealcms import create_app
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+
+
+def app_with_pydot():
+    """Create the test app, including the pydot extension."""
+    return create_app(instance_path=os.path.join(HERE, 'instance'),
+                      test_config={'MARKDOWN_EXTENSIONS': ['incorporealcms.mdx.pydot']})
+
+
+def test_functional_initialization():
+    """Test initialization with the graphviz config."""
+    app = app_with_pydot()
+    assert app is not None
+
+
+def test_graphviz_is_rendered():
+    """Initialize the app with the graphviz extension and ensure it does something."""
+    app = app_with_pydot()
+    client = app.test_client()
+
+    response = client.get('/test-graphviz')
+    assert response.status_code == 200
+    assert b'~~~pydot' not in response.data
+    assert b'data:image/png;base64' in response.data
+
+
+def test_two_graphviz_are_rendered():
+    """Test two images are rendered."""
+    app = app_with_pydot()
+    client = app.test_client()
+
+    response = client.get('/test-two-graphviz')
+    assert response.status_code == 200
+    assert b'~~~pydot' not in response.data
+    assert b'data:image/png;base64' in response.data
+
+
+def test_invalid_graphviz_is_not_rendered():
+    """Check that invalid graphviz doesn't blow things up."""
+    app = app_with_pydot()
+    client = app.test_client()
+
+    response = client.get('/test-invalid-graphviz')
+    assert response.status_code == 500
+    assert b'INTERNAL SERVER ERROR' in response.data
+
+
+def test_figures_are_rendered(client):
+    """Test that a page with my figure syntax renders as expected."""
+    response = client.get('/figures')
+    assert response.status_code == 200
+    assert (b'<figure class="right"><img alt="fancy captioned logo" src="bss-square-no-bg.png" />'
+            b'<figcaption>this is my cool logo!</figcaption></figure>') in response.data
+    assert (b'<figure><img alt="vanilla captioned logo" src="bss-square-no-bg.png" />'
+            b'<figcaption>this is my cool logo without an attr!</figcaption>\n</figure>') in response.data
+    assert (b'<figure class="left"><img alt="fancy logo" src="bss-square-no-bg.png" />'
+            b'<span></span></figure>') in response.data
+    assert b'<figure><img alt="just a logo" src="bss-square-no-bg.png" /></figure>' in response.data

tests/functional_tests.py

@@ -1,33 +1,122 @@
 """Test page requests."""
 import re
+from unittest.mock import patch
 
 
 def test_page_that_exists(client):
     """Test that the app can serve a basic file at the index."""
     response = client.get('/')
     assert response.status_code == 200
-    assert b'<h1>test index</h1>' in response.data
+    assert b'<h1 id="test-index">test index</h1>' in response.data
+
+
+def test_direct_file_that_exists(client):
+    """Test that the app can serve a basic file at the index."""
+    response = client.get('/foo.txt')
+    assert response.status_code == 200
+    assert b'test file' in response.data
 
 
 def test_page_that_doesnt_exist(client):
-    """Test that the app returns 404 for nonsense requests."""
+    """Test that the app returns 404 for nonsense requests and they use my error page."""
     response = client.get('/ohuesthaoeusth')
     assert response.status_code == 404
+    assert b'<b><tt>/ohuesthaoeusth</tt></b> does not seem to exist' in response.data
+    # test the contact email config
+    assert b'admin@example.com' in response.data
+
+
+def test_files_outside_pages_do_not_get_served(client):
+    """Test that page pathing doesn't break out of the instance/pages/ dir, and the error uses my error page."""
+    response = client.get('/../unreachable')
+    assert response.status_code == 400
+    assert b'You\'re doing something you\'re not supposed to. Stop it?' in response.data
+
+
+def test_internal_server_error_serves_error_page(client):
+    """Test that various exceptions serve up the 500 page."""
+    response = client.get('/actually-a-png')
+    assert response.status_code == 500
+    assert b'INTERNAL SERVER ERROR' in response.data
+    # test the contact email config
+    assert b'admin@example.com' in response.data
+
+
+def test_oserror_is_500(client, app):
+    """Test that an OSError raises as a 500."""
+    with app.test_request_context():
+        with patch('flask.current_app.open_instance_resource', side_effect=OSError):
+            response = client.get('/')
+            assert response.status_code == 500
+            assert b'INTERNAL SERVER ERROR' in response.data
+
+
+def test_unsupported_file_type_is_500(client, app):
+    """Test a coding condition mishap raises as a 500."""
+    with app.test_request_context():
+        with patch('incorporealcms.pages.request_path_to_instance_resource_path', return_value=('foo', 'bar')):
+            response = client.get('/')
+            assert response.status_code == 500
+            assert b'INTERNAL SERVER ERROR' in response.data
+
+
+def test_weird_paths_do_not_get_served(client):
+    """Test that we clean up requests as desired."""
+    response = client.get('/../../')
+    assert response.status_code == 400
 
 
 def test_page_with_title_metadata(client):
     """Test that a page with title metadata has its title written."""
     response = client.get('/')
     assert response.status_code == 200
-    assert b'<title>Index - incorporeal.org</title>' in response.data
+    assert b'<title>Index - example.com</title>' in response.data
 
 
 def test_page_without_title_metadata(client):
     """Test that a page without title metadata gets the default title."""
     response = client.get('/no-title')
     assert response.status_code == 200
-    assert b'<title>incorporeal.org</title>' in response.data
+    assert b'<title>/no-title - example.com</title>' in response.data
-    assert b'<h1>this page doesn\'t have a title!</h1>' in response.data
+
+
+def test_page_in_subdir_without_title_metadata(client):
+    """Test that the title-less page display is as expected."""
+    response = client.get('/subdir//page-no-title')
+    assert response.status_code == 200
+    assert b'<title>/subdir/page-no-title - example.com</title>' in response.data
+
+
+def test_page_with_card_metadata(client):
+    """Test that a page with opengraph metadata."""
+    response = client.get('/more-metadata')
+    assert response.status_code == 200
+    assert b'<meta property="og:title" content="title for the page - example.com">' in response.data
+    assert b'<meta property="og:description" content="description of this page made even longer">' in response.data
+    assert b'<meta property="og:image" content="http://buh.com/test.img">' in response.data
+
+
+def test_page_with_card_title_even_when_no_metadata(client):
+    """Test that a page without metadata still has a card with the derived title."""
+    response = client.get('/no-title')
+    assert response.status_code == 200
+    assert b'<meta property="og:title" content="/no-title - example.com">' in response.data
+    assert b'<meta property="og:description"' not in response.data
+    assert b'<meta property="og:image"' not in response.data
+
+
+def test_page_with_forced_empty_title_just_shows_suffix(client):
+    """Test that if a page specifies a blank Title meta tag explicitly, only the suffix is used in the title."""
+    response = client.get('/forced-no-title')
+    assert response.status_code == 200
+    assert b'<title>example.com</title>' in response.data
+
+
+def test_page_with_redirect_meta_url_redirects(client):
+    """Test that if a page specifies a URL to redirect to, that the site serves up a 301."""
+    response = client.get('/redirect')
+    assert response.status_code == 301
+    assert response.location == 'http://www.google.com/'
 
 
 def test_page_has_modified_timestamp(client):
@@ -45,6 +134,44 @@ def test_that_page_request_redirects_to_directory(client):
     """
     response = client.get('/subdir')
     assert response.status_code == 301
+    assert response.location == '/subdir/'
+
+
+def test_that_request_to_symlink_redirects_markdown(client):
+    """Test that a request to /foo redirects to /what-foo-points-at."""
+    response = client.get('/symlink-to-no-title')
+    assert response.status_code == 301
+    assert response.location == '/no-title'
+
+
+def test_that_request_to_symlink_redirects_file(client):
+    """Test that a request to /foo.txt redirects to /what-foo-points-at.txt."""
+    response = client.get('/symlink-to-foo.txt')
+    assert response.status_code == 301
+    assert response.location == '/foo.txt'
+
+
+def test_that_request_to_symlink_redirects_directory(client):
+    """Test that a request to /foo/ redirects to /what-foo-points-at/."""
+    response = client.get('/symlink-to-subdir/')
+    assert response.status_code == 301
+    assert response.location == '/subdir'
+    # sadly, this location also redirects
+    response = client.get('/subdir')
+    assert response.status_code == 301
+    assert response.location == '/subdir/'
+    # but we do get there
+    response = client.get('/subdir/')
+    assert response.status_code == 200
+
+
+def test_that_request_to_symlink_redirects_subdirectory(client):
+    """Test that a request to /foo/bar redirects to /what-foo-points-at/bar."""
+    response = client.get('/symlink-to-subdir/page-no-title')
+    assert response.status_code == 301
+    assert response.location == '/subdir/page-no-title'
+    response = client.get('/subdir/page-no-title')
+    assert response.status_code == 200
 
 
 def test_that_dir_request_does_not_redirect(client):
@@ -52,3 +179,62 @@ def test_that_dir_request_does_not_redirect(client):
     response = client.get('/subdir/')
     assert response.status_code == 200
     assert b'another page' in response.data
+
+
+def test_setting_selected_style_includes_cookie(client):
+    """Test that a request with style=foo sets the cookie and renders appropriately."""
+    response = client.get('/')
+    style_cookie = next((cookie for cookie in client.cookie_jar if cookie.name == 'user-style'), None)
+    assert style_cookie is None
+
+    response = client.get('/?style=light')
+    style_cookie = next((cookie for cookie in client.cookie_jar if cookie.name == 'user-style'), None)
+    assert response.status_code == 200
+    assert b'/static/css/light.css' in response.data
+    assert b'/static/css/dark.css' not in response.data
+    assert style_cookie.value == 'light'
+
+    response = client.get('/?style=dark')
+    style_cookie = next((cookie for cookie in client.cookie_jar if cookie.name == 'user-style'), None)
+    assert response.status_code == 200
+    assert b'/static/css/dark.css' in response.data
+    assert b'/static/css/light.css' not in response.data
+    assert style_cookie.value == 'dark'
+
+
+def test_pages_can_supply_alternate_templates(client):
+    """Test that pages can supply templates other than the default."""
+    response = client.get('/')
+    assert b'class="site-wrap site-wrap-normal-width"' in response.data
+    assert b'class="site-wrap site-wrap-double-width"' not in response.data
+    response = client.get('/custom-template')
+    assert b'class="site-wrap site-wrap-normal-width"' not in response.data
+    assert b'class="site-wrap site-wrap-double-width"' in response.data
+
+
+def test_extra_footer_per_page(client):
+    """Test that we don't include the extra-footer if there isn't one (or do if there is)."""
+    response = client.get('/')
+    assert b'<div class="extra-footer">' not in response.data
+    response = client.get('/index-but-with-footer')
+    assert b'<div class="extra-footer"><i>ooo <a href="a">a</a></i>' in response.data
+
+
+def test_serving_static_files(client):
+    """Test the usage of send_from_directory to serve extra static files."""
+    response = client.get('/custom-static/css/warm.css')
+    assert response.status_code == 200
+
+    # can't serve directories, just files
+    response = client.get('/custom-static/')
+    assert response.status_code == 404
+    response = client.get('/custom-static/css/')
+    assert response.status_code == 404
+    response = client.get('/custom-static/css')
+    assert response.status_code == 404
+
+    # can't serve files that don't exist or bad paths
+    response = client.get('/custom-static/css/cold.css')
+    assert response.status_code == 404
+    response = client.get('/custom-static/css/../../unreachable.md')
+    assert response.status_code == 404

tests/instance/config.py

@@ -15,6 +15,10 @@ LOGGING = {
         },
     },
     'loggers': {
+        'incorporealcms.mdx': {
+            'level': 'DEBUG',
+            'handlers': ['console'],
+        },
         'incorporealcms.pages': {
             'level': 'DEBUG',
             'handlers': ['console'],

tests/instance/custom-static/css/warm.css

@@ -0,0 +1,3 @@
+* {
+    color: red;
+}

tests/instance/pages/custom-template.md

@@ -0,0 +1,3 @@
+Template:           base-wide.html
+
+testttttttttt

tests/instance/pages/figures.md

@@ -0,0 +1,13 @@
+# test of figures
+
+|> ![fancy captioned logo](bss-square-no-bg.png)
+|: this is my cool logo!
+{: .right }
+
+|> ![vanilla captioned logo](bss-square-no-bg.png)
+|: this is my cool logo without an attr!
+
+|> ![fancy logo](bss-square-no-bg.png)
+{: .left }
+
+|> ![just a logo](bss-square-no-bg.png)

tests/instance/pages/foo.txt

@@ -0,0 +1 @@
+test file

tests/instance/pages/forced-no-title.md

@@ -0,0 +1,3 @@
+Title:
+
+some words are here

tests/instance/pages/index-but-with-footer.md

@@ -0,0 +1,6 @@
+Title:      Index
+Footer:     ooo <a href="a">a</a>
+
+# test index
+
+this is some test content

tests/instance/pages/more-metadata.md

@@ -0,0 +1,6 @@
+Title:          title for the page
+Description:    description of this page
+                made even longer
+Image:          http://buh.com/test.img
+
+hello

tests/instance/pages/no-index-dir/page.md

@@ -0,0 +1 @@
+this is a test page

tests/instance/pages/redirect.md

@@ -0,0 +1 @@
+Redirect:           http://www.google.com/

tests/instance/pages/subdir/page-no-title.md

@@ -0,0 +1 @@
+this is just a page

tests/instance/pages/symlink-to-foo.txt

@@ -0,0 +1 @@
+foo.txt

tests/instance/pages/symlink-to-no-title.md

@@ -0,0 +1 @@
+no-title.md

tests/instance/pages/symlink-to-subdir

@@ -0,0 +1 @@
+subdir

tests/instance/pages/test-graphviz.md

@@ -0,0 +1,12 @@
+# test
+
+test
+~~~{ pydot:attack-plan }
+digraph G {
+    rankdir=LR
+    Earth
+    Mars
+    Earth -> Mars
+}
+~~~
+more test

tests/instance/pages/test-invalid-graphviz.md

@@ -0,0 +1,11 @@
+# test
+
+test
+~~~{ pydot:attack-plan }
+    rankdir=LR
+    Earth
+    Mars
+    Earth -> Mars
+}
+~~~
+more test

tests/instance/pages/test-two-graphviz.md

@@ -0,0 +1,25 @@
+# test
+
+test
+
+~~~{ pydot:attack-plan }
+digraph G {
+    rankdir=LR
+    Earth
+    Mars
+    Earth -> Mars
+}
+~~~
+
+more test
+
+~~~{ pydot:new-attack-plan }
+digraph H {
+    rankdir=LR
+    Venus
+    Mars
+    Venus -> Mars
+}
+~~~
+
+done

tests/instance/unreachable.md

@@ -0,0 +1 @@
+this file exists but the app should not serve it.

tests/test_factory.py

@@ -20,23 +20,27 @@ def test_markdown_meta_extension_always():
     client = app.test_client()
     response = client.get('/')
     assert response.status_code == 200
-    assert b'<title>Index - incorporeal.org</title>' in response.data
+    assert b'<title>Index - example.com</title>' in response.data
 
 
-def test_extra_markdown_extensions_work():
+def test_custom_markdown_extensions_work():
-    """Test we can load more extensions via config, and that they work."""
+    """Test we can change extensions via config, and that they work.
+
+    This used to test smarty, but that's added by default now, so we test
+    that it can be removed by overriding the option.
+    """
     app = create_app(instance_path=os.path.join(HERE, 'instance'))
     client = app.test_client()
     response = client.get('/mdash-or-triple-dash')
     assert response.status_code == 200
-    assert b'word --- word' in response.data
+    assert b'word &mdash; word' in response.data
 
     app = create_app(instance_path=os.path.join(HERE, 'instance'),
-                     test_config={'MARKDOWN_EXTENSIONS': ['smarty']})
+                     test_config={'MARKDOWN_EXTENSIONS': []})
     client = app.test_client()
     response = client.get('/mdash-or-triple-dash')
     assert response.status_code == 200
-    assert b'word &mdash; word' in response.data
+    assert b'word --- word' in response.data
 
 
 def test_title_override():
@@ -46,7 +50,7 @@ def test_title_override():
     client = app.test_client()
     response = client.get('/no-title')
     assert response.status_code == 200
-    assert b'<title>suou.net</title>' in response.data
+    assert b'<title>/no-title - suou.net</title>' in response.data
 
 
 def test_media_file_access(client):
@@ -54,3 +58,22 @@ def test_media_file_access(client):
     response = client.get('/media/favicon.png')
     assert response.status_code == 200
     assert response.headers['content-type'] == 'image/png'
+
+
+def test_favicon_override():
+    """Test that a configuration with a specific favicon overrides the default."""
+    instance_path = os.path.join(HERE, 'instance')
+    app = create_app(instance_path=instance_path, test_config={'FAVICON': '/media/foo.png'})
+    client = app.test_client()
+    response = client.get('/no-title')
+    assert response.status_code == 200
+    assert b'<link rel="icon" href="/media/foo.png">' in response.data
+
+
+def test_misconfigured_markdown_extensions():
+    """Test that a misconfigured markdown extensions leads to a 500 at render time."""
+    instance_path = os.path.join(HERE, 'instance')
+    app = create_app(instance_path=instance_path, test_config={'MARKDOWN_EXTENSIONS': 'WRONG'})
+    client = app.test_client()
+    response = client.get('/no-title')
+    assert response.status_code == 500

tests/test_pages.py

@@ -1,100 +1,278 @@
 """Unit test helper methods."""
+import os
+
+import pytest
 from werkzeug.http import dump_cookie
 
-from incorporealcms.pages import generate_parent_navs, is_file_path_actually_dir_path, render, resolve_page_file
+from incorporealcms import create_app
+from incorporealcms.pages import (generate_parent_navs, instance_resource_path_to_request_path, render,
+                                  request_path_to_breadcrumb_display, request_path_to_instance_resource_path)
 
-
+HERE = os.path.dirname(os.path.abspath(__file__))
-def test_resolve_page_file_dir_to_index():
-    """Test that a request to a directory path results in the dir's index.md."""
-    assert resolve_page_file('foo/') == 'pages/foo/index.md'
-
-
-def test_resolve_page_file_subdir_to_index():
-    """Test that a request to a dir's subdir path results in the subdir's index.md."""
-    assert resolve_page_file('foo/bar/') == 'pages/foo/bar/index.md'
-
-
-def test_resolve_page_file_other_requests_fine():
-    """Test that a request to non-dir path results in a Markdown file."""
-    assert resolve_page_file('foo/baz') == 'pages/foo/baz.md'
 
 
 def test_generate_page_navs_index(app):
     """Test that the index page has navs to the root (itself)."""
     with app.app_context():
-        assert generate_parent_navs('/') == [('incorporeal.org', '/')]
+        assert generate_parent_navs('pages/index.md') == [('example.com', '/')]
-
-
-def test_generate_page_navs_alternate_index(app):
-    """Test that the index page (as a page, not a path) also has navs only to the root (by path)."""
-    with app.app_context():
-        assert generate_parent_navs('index') == [('incorporeal.org', '/')]
 
 
 def test_generate_page_navs_subdir_index(app):
     """Test that dir pages have navs to the root and themselves."""
     with app.app_context():
-        assert generate_parent_navs('subdir/') == [('incorporeal.org', '/'), ('/subdir/', '/subdir/')]
+        assert generate_parent_navs('pages/subdir/index.md') == [('example.com', '/'), ('subdir', '/subdir/')]
 
 
 def test_generate_page_navs_subdir_real_page(app):
     """Test that real pages have navs to the root, their parent, and themselves."""
     with app.app_context():
-        assert generate_parent_navs('subdir/page') == [('incorporeal.org', '/'), ('/subdir/', '/subdir/'),
+        assert generate_parent_navs('pages/subdir/page.md') == [('example.com', '/'), ('subdir', '/subdir/'),
                                                                 ('Page', '/subdir/page')]
 
 
 def test_generate_page_navs_subdir_with_title_parsing_real_page(app):
     """Test that title metadata is used in the nav text."""
     with app.app_context():
-        assert generate_parent_navs('subdir-with-title/page') == [
+        assert generate_parent_navs('pages/subdir-with-title/page.md') == [
-            ('incorporeal.org', '/'),
+            ('example.com', '/'),
             ('SUB!', '/subdir-with-title/'),
-            ('/subdir-with-title/page', '/subdir-with-title/page')
+            ('page', '/subdir-with-title/page')
         ]
 
 
-def test_is_file_path_actually_dir_path_valid_file_is_yes(app):
+def test_generate_page_navs_subdir_with_no_index(app):
-    """Test that a file request for what's actually a directory is detected as such."""
+    """Test that breadcrumbs still generate even if a subdir doesn't have an index.md."""
     with app.app_context():
-        assert is_file_path_actually_dir_path('/subdir')
+        assert generate_parent_navs('pages/no-index-dir/page.md') == [
-
+            ('example.com', '/'),
-
+            ('/no-index-dir/', '/no-index-dir/'),
-def test_is_file_path_actually_dir_path_valid_dir_is_no(app):
+            ('page', '/no-index-dir/page')
-    """Test that a directory request is still a directory request."""
+        ]
-    with app.app_context():
-        assert not is_file_path_actually_dir_path('/subdir/')
-
-
-def test_is_file_path_actually_dir_path_nonsense_file_is_no(app):
-    """Test that requests to nonsense file-looking paths aren't treated as dirs."""
-    with app.app_context():
-        assert not is_file_path_actually_dir_path('/antphnathpnthapnthsnthax')
-
-
-def test_is_file_path_actually_dir_path_nonsense_dir_is_no(app):
-    """Test that a directory request is a directory request even if the dir doesn't exist."""
-    with app.app_context():
-        assert not is_file_path_actually_dir_path('/antphnathpnthapnthsnthax/')
 
 
 def test_render_with_user_dark_theme(app):
     """Test that a request with the dark theme selected renders the dark theme."""
     cookie = dump_cookie("user-style", 'dark')
     with app.test_request_context(headers={'COOKIE': cookie}):
-        assert b'dark.css' in render('base.html').data
+        assert b'/static/css/dark.css' in render('base.html').data
-        assert b'light.css' not in render('base.html').data
+        assert b'/static/css/light.css' not in render('base.html').data
 
 
 def test_render_with_user_light_theme(app):
     """Test that a request with the light theme selected renders the light theme."""
     with app.test_request_context():
-        assert b'light.css' in render('base.html').data
+        assert b'/static/css/light.css' in render('base.html').data
-        assert b'dark.css' not in render('base.html').data
+        assert b'/static/css/dark.css' not in render('base.html').data
 
 
 def test_render_with_no_user_theme(app):
     """Test that a request with no theme set renders the light theme."""
     with app.test_request_context():
-        assert b'light.css' in render('base.html').data
+        assert b'/static/css/light.css' in render('base.html').data
-        assert b'dark.css' not in render('base.html').data
+        assert b'/static/css/dark.css' not in render('base.html').data
+
+
+def test_render_with_theme_defaults_affects_html(app):
+    """Test that the base themes are all that's presented in the HTML."""
+    # test we can remove stuff from the default
+    with app.test_request_context():
+        assert b'?style=light' in render('base.html').data
+        assert b'?style=dark' in render('base.html').data
+        assert b'?style=plain' in render('base.html').data
+
+
+def test_render_with_theme_overrides_affects_html(app):
+    """Test that the overridden themes are presented in the HTML."""
+    # test we can remove stuff from the default
+    restyled_app = create_app(instance_path=os.path.join(HERE, 'instance'),
+                              test_config={'PAGE_STYLES': {'light': '/static/css/light.css'}})
+    with restyled_app.test_request_context():
+        assert b'?style=light' in render('base.html').data
+        assert b'?style=dark' not in render('base.html').data
+        assert b'?style=plain' not in render('base.html').data
+
+    # test that we can add new stuff too/instead
+    restyled_app = create_app(instance_path=os.path.join(HERE, 'instance'),
+                              test_config={'PAGE_STYLES': {'cool': '/static/css/cool.css',
+                                                           'warm': '/static/css/warm.css'},
+                                           'DEFAULT_PAGE_STYLE': 'warm'})
+    with restyled_app.test_request_context():
+        assert b'?style=cool' in render('base.html').data
+        assert b'?style=warm' in render('base.html').data
+
+
+def test_render_with_theme_overrides(app):
+    """Test that the loaded themes can be overridden from the default."""
+    cookie = dump_cookie("user-style", 'cool')
+    restyled_app = create_app(instance_path=os.path.join(HERE, 'instance'),
+                              test_config={'PAGE_STYLES': {'cool': '/static/css/cool.css',
+                                                           'warm': '/static/css/warm.css'}})
+    with restyled_app.test_request_context(headers={'COOKIE': cookie}):
+        assert b'/static/css/cool.css' in render('base.html').data
+        assert b'/static/css/warm.css' not in render('base.html').data
+
+
+def test_render_with_theme_overrides_not_found_is_default(app):
+    """Test that theme overrides work, and if a requested theme doesn't exist, the default is loaded."""
+    cookie = dump_cookie("user-style", 'nonexistent')
+    restyled_app = create_app(instance_path=os.path.join(HERE, 'instance'),
+                              test_config={'PAGE_STYLES': {'cool': '/static/css/cool.css',
+                                                           'warm': '/static/css/warm.css'},
+                                           'DEFAULT_PAGE_STYLE': 'warm'})
+    with restyled_app.test_request_context(headers={'COOKIE': cookie}):
+        assert b'/static/css/warm.css' in render('base.html').data
+        assert b'/static/css/nonexistent.css' not in render('base.html').data
+
+
+def test_request_path_to_instance_resource_path(app):
+    """Test a normal URL request is transformed into the file path."""
+    with app.test_request_context():
+        assert request_path_to_instance_resource_path('index') == ('pages/index.md', 'markdown')
+
+
+def test_request_path_to_instance_resource_path_direct_file(app):
+    """Test a normal URL request is transformed into the file path."""
+    with app.test_request_context():
+        assert request_path_to_instance_resource_path('no-title') == ('pages/no-title.md', 'markdown')
+
+
+def test_request_path_to_instance_resource_path_in_subdir(app):
+    """Test a normal URL request is transformed into the file path."""
+    with app.test_request_context():
+        assert request_path_to_instance_resource_path('subdir/page') == ('pages/subdir/page.md', 'markdown')
+
+
+def test_request_path_to_instance_resource_path_subdir_index(app):
+    """Test a normal URL request is transformed into the file path."""
+    with app.test_request_context():
+        assert request_path_to_instance_resource_path('subdir/') == ('pages/subdir/index.md', 'markdown')
+
+
+def test_request_path_to_instance_resource_path_relatives_walked(app):
+    """Test a normal URL request is transformed into the file path."""
+    with app.test_request_context():
+        assert (request_path_to_instance_resource_path('subdir/more-subdir/../../more-metadata') ==
+                ('pages/more-metadata.md', 'markdown'))
+
+
+def test_request_path_to_instance_resource_path_relatives_walked_indexes_work_too(app):
+    """Test a normal URL request is transformed into the file path."""
+    with app.test_request_context():
+        assert request_path_to_instance_resource_path('subdir/more-subdir/../../') == ('pages/index.md', 'markdown')
+
+
+def test_request_path_to_instance_resource_path_relatives_walked_into_subdirs_also_fine(app):
+    """Test a normal URL request is transformed into the file path."""
+    with app.test_request_context():
+        assert (request_path_to_instance_resource_path('subdir/more-subdir/../../subdir/page') ==
+                ('pages/subdir/page.md', 'markdown'))
+
+
+def test_request_path_to_instance_resource_path_permission_error_on_ref_above_pages(app):
+    """Test that attempts to get above the base dir ("/../../foo") fail."""
+    with app.test_request_context():
+        with pytest.raises(PermissionError):
+            assert request_path_to_instance_resource_path('../unreachable')
+
+
+def test_request_path_to_instance_resource_path_isadirectory_on_file_like_req_for_dir(app):
+    """Test that a request for e.g. '/foo' when foo is a dir indicate to redirect."""
+    with app.test_request_context():
+        with pytest.raises(IsADirectoryError):
+            assert request_path_to_instance_resource_path('subdir')
+
+
+def test_request_path_to_instance_resource_path_actual_file(app):
+    """Test that a request for e.g. '/foo.png' when foo.png is a real file works."""
+    with app.test_request_context():
+        assert (request_path_to_instance_resource_path('bss-square-no-bg.png') ==
+                ('pages/bss-square-no-bg.png', 'file'))
+
+
+def test_request_path_to_instance_resource_path_markdown_symlink(app):
+    """Test that a request for e.g. '/foo' when foo.md is a symlink to another .md file redirects."""
+    with app.test_request_context():
+        assert (request_path_to_instance_resource_path('symlink-to-no-title') ==
+                ('pages/no-title.md', 'symlink'))
+
+
+def test_request_path_to_instance_resource_path_file_symlink(app):
+    """Test that a request for e.g. '/foo' when foo.txt is a symlink to another .txt file redirects."""
+    with app.test_request_context():
+        assert (request_path_to_instance_resource_path('symlink-to-foo.txt') ==
+                ('pages/foo.txt', 'symlink'))
+
+
+def test_request_path_to_instance_resource_path_dir_symlink(app):
+    """Test that a request for e.g. '/foo' when /foo is a symlink to /bar redirects."""
+    with app.test_request_context():
+        assert (request_path_to_instance_resource_path('symlink-to-subdir/') ==
+                ('pages/subdir', 'symlink'))
+
+
+def test_request_path_to_instance_resource_path_subdir_symlink(app):
+    """Test that a request for e.g. '/foo/baz' when /foo is a symlink to /bar redirects."""
+    with app.test_request_context():
+        assert (request_path_to_instance_resource_path('symlink-to-subdir/page-no-title') ==
+                ('pages/subdir/page-no-title.md', 'symlink'))
+
+
+def test_request_path_to_instance_resource_path_nonexistant_file_errors(app):
+    """Test that a request for something not on disk errors."""
+    with app.test_request_context():
+        with pytest.raises(FileNotFoundError):
+            assert request_path_to_instance_resource_path('nthanpthpnh')
+
+
+def test_request_path_to_instance_resource_path_absolute_file_errors(app):
+    """Test that a request for something not on disk errors."""
+    with app.test_request_context():
+        with pytest.raises(PermissionError):
+            assert request_path_to_instance_resource_path('/etc/hosts')
+
+
+def test_instance_resource_path_to_request_path_on_index(app):
+    """Test index.md -> /."""
+    with app.test_request_context():
+        assert instance_resource_path_to_request_path('index.md') == ''
+
+
+def test_instance_resource_path_to_request_path_on_page(app):
+    """Test no-title.md -> no-title."""
+    with app.test_request_context():
+        assert instance_resource_path_to_request_path('no-title.md') == 'no-title'
+
+
+def test_instance_resource_path_to_request_path_on_subdir(app):
+    """Test subdir/index.md -> subdir/."""
+    with app.test_request_context():
+        assert instance_resource_path_to_request_path('subdir/index.md') == 'subdir/'
+
+
+def test_instance_resource_path_to_request_path_on_subdir_and_page(app):
+    """Test subdir/page.md -> subdir/page."""
+    with app.test_request_context():
+        assert instance_resource_path_to_request_path('subdir/page.md') == 'subdir/page'
+
+
+def test_request_resource_request_root(app):
+    """Test that a request can resolve to a resource and back to a request."""
+    with app.test_request_context():
+        instance_path, _ = request_path_to_instance_resource_path('index')
+        instance_resource_path_to_request_path(instance_path) == ''
+
+
+def test_request_resource_request_page(app):
+    """Test that a request can resolve to a resource and back to a request."""
+    with app.test_request_context():
+        instance_path, _ = request_path_to_instance_resource_path('no-title')
+        instance_resource_path_to_request_path(instance_path) == 'no-title'
+
+
+def test_request_path_to_breadcrumb_display_patterns():
+    """Test various conversions from request path to leaf nodes for display in the breadcrumbs."""
+    assert request_path_to_breadcrumb_display('/foo') == 'foo'
+    assert request_path_to_breadcrumb_display('/foo/') == 'foo'
+    assert request_path_to_breadcrumb_display('/foo/bar') == 'bar'
+    assert request_path_to_breadcrumb_display('/foo/bar/') == 'bar'
+    assert request_path_to_breadcrumb_display('/') == ''

tox.ini

@@ -4,7 +4,7 @@
 # and then run "tox" from this directory.
 
 [tox]
-envlist = begin,py37,py38,coverage,security,lint,bundle
+envlist = begin,py38,py39,py310,coverage,security,lint,bundle
 
 [testenv]
 # build a wheel and test it
@@ -31,12 +31,17 @@ deps = setuptools
 skip_install = true
 commands = coverage erase
 
-[testenv:py37]
+[testenv:py38]
 # run pytest with coverage
 commands =
     pytest --cov-append --cov={envsitepackagesdir}/incorporealcms/ --cov-branch
 
-[testenv:py38]
+[testenv:py39]
+# run pytest with coverage
+commands =
+    pytest --cov-append --cov={envsitepackagesdir}/incorporealcms/ --cov-branch
+
+[testenv:py310]
 # run pytest with coverage
 commands =
     pytest --cov-append --cov={envsitepackagesdir}/incorporealcms/ --cov-branch
@@ -51,8 +56,10 @@ commands =
 # run security checks
 #
 # again it seems the most valuable here to run against the packaged code
+# 51457 is nearly a red herring that I'm stuck with because tox is pinned, try removing occasionally
 commands =
     bandit {envsitepackagesdir}/incorporealcms/ -r
+    safety check -r requirements/requirements-dev.txt -i 51457
 
 [testenv:lint]
 # run style checks
@@ -74,12 +81,6 @@ source =
 [coverage:run]
 branch = True
 
-# redundant with pytest --cov above, but this tricks the coverage.xml report into
-# using the full path, otherwise files with the same name in different paths
-# get clobbered. maybe appends would fix this, IDK
-include =
-    .tox/**/incorporealcms/
-
 omit =
     **/_version.py