build_site.py
While making this website, I found myself in need of a static site builder. Writing html by hand is possible, but some content (like blogposts) quickly becomes a chore to edit in html form. There are many site builders available, but I found all of them unsatisfactory for one of the following reasons:
So I made my own: build_site.py ~600 lines. It is simple to use while also being easily configurable to do exactly what you want via python. Okay, technically every open source application is 'configurable' if you think of changing the source code as 'configuration', but this script is made with this in mind, and is based on the understanding the best static site builder is actually just python. In fact, you can write python code wrapped in {%% %%}
and use echo()
calls to build the html, similar to php files. The only real assumption it makes is that your html template filenames are prefixed with an underscore, though even that can be turned off.
There are two ways this script can be used:
By default, it reads all html and php files starting with an underscore, applies the template, and outputs the built file without the underscore to the same directory. You can override this by setting the --input
and --output
options, or by setting the %output
key in the frontmatter at the start of the template file to output to a custom path.
Details
python build_site.py path/to/project
/path/to/project
is taken as the 'root' path of the project. The script can also take an -i
or --input
argument (among others) to only process a subdirectory of root. The reason why there are two input-related arguments is to allow the use of absolute paths in templates, similarly to how they can be used on the website, e.g /data/file.md
.
To process only a single template, the --input
can be set to the filepath, or, if the template does not use absolute paths, you can just provide the filepath as the root argument.
{{item.key}}
Substituted with the corresponding key from the frontmatter of the file. {{item.content}}
stores the preprocessed contents of the file (usually, depending on the preprocessor). The default markdown preprocessor also adds a few other keys such as item.firstheader and item.filename, see the code for details.
{{function(input)}}
Custom function. Substituted with the output of the custom function defined in the Functions class. input
can be the item name, e.g 'item' (passes the python dict), 'item.key' (value of the corresponding frontmatter property), or *%* (all data dict). Comma separated inputs will be split and passed as separate arguments. Any other input will be passed to the function as a string.
{% for item in path %}..{% endfor %}
For loop. Contents are added for every item in path. By default, path
can be either a folder or a csv file, though it's easy to extend the script with a custom iterator (see the extending section).
Example
{% if cond %}..{% elif cond2 %}..{% else %}..{% endif %}
If condition. Contents are added if cond
evaluates to true. cond
is a python expression.
Example
{{cond ? trueVal : falseVal}}
Shorthand if. cond
is a python expression. If true, trueVal is substituted, else falseVal.
Example
{% import path %}, {% import path as item %}
Pastes the file contents of path
after preprocessing. When using with as
, saves the file data into the item
variable instead of pasting the contents, after which the keys will be accessible like usual with {{item.key}}
.
Example
{% export item as template_path to output_path %}
Create an html file from an item
and template. Can be used in place of the %foreach frontmatter property.
Example
{%% python code %%}
Use python and echo()
calls to build the html.
Example
There are two special keys which can be defined in the frontmatter of any template file:
%output: path
- Overrides the default output path. Set to %none
to ignore the file.%foreach: path
- Special directive that will create a new html for every item at the path. The html files will be output to the path defined in %output
. The keys can be accessed via page
, e.g {{page.content}}
. See the example.There are three classes defined at the start of build_site.py:
Preprocessors
- Defines functions for preprocessing different file formats. The field extdict
maps file extensions to the appropriate preprocessor function. By default, markdown files are converted to html, and html files are pasted as they are. To define a custom preprocessor, write a function taking a path and returning a dict and add the corresponding file extension to extdict
.Iterators
- Defines iterators for {% for %} loops. extdict
maps file extensions to the appropriate iterator. There is an iterator for folders and one for csv files by default. To define a custom iterator, write a function taking a path and returning a list of dicts, and add the corresponding file extension to extdict
.Functions
- Defines custom functions for use inside {{}}. To define a custom function simply write a new function returning a string in the class. The default function py
will evaluate the passed argument as a python expression and return the result. Of course, the functions themselves can be set up to do anything – for example, for loops, if conditions and the import and export directives can be easily implemented as functions.For preprocessors and iterators you can also define functions that modify the input in a certain way. To do this, write a function taking a string (which will be the contents of the parentheses) and returning a dict or list of dicts, and add the key functionName(*)
to extdict. For preprocessors, the function raw
can be used to import files without preprocessing, e.g {% import raw(/path/file.md) %}
. For iterators, the function reverse
can be used to reverse any list, e.g {% for file in reverse(/path/folder) %}
. The function range
only takes two numbers as input and returns a list where the elements are dicts of the form {'index':num}, e.g {% for x in range(1,10) %}Number:{{x.index}}{% endfor %}
<div class="index-links">
{% for item in /data/blog %}
<a href="#{{item.id}}">{{item.title}}</a>
{% endfor %}
</div>
This will iterate over the rows of a csv file in reverse and only add the ones where the 'hidden' column is not 'true':
{% for row in reverse(/data/links.csv) %}
{{row.hidden != 'true' ? <a href="#{{row.url}}">{{row.text}}</a> : }}
{% endfor %}
or, with python:
{%%
for row in reversed(Iterators.get_data('/data/links.csv')):
if row.hidden != 'true':
echo(f'<a href="{item.id}">{item.title}</a>')
%%}
Note that
Iterators.get_data('reverse(/data/links.csv)')
= Iterators.reverse('/data/links.csv')
= reversed(Iterators.get_data('/data/links.csv'))
= reversed(Iterators.csv(get_real_path('/data/links.csv')))
Create a simple blog index page showing the first three lines of every post:
{% for item in /data/blog %}
<input id="{{item.id}}-check" type="checkbox" {{'hidden' not in item.classes ? checked : }}>
<div id="{{item.id}}" class="post {{item.classes}}">
<h1>{{item.title}}</h1>
<span>
<a href="{{item.id}}.html">View post</a>
<span>Posted: {{item.date}}</span>
<span>{{wordCount(item.content)}} words</span>
</span>
<div class="contents">
{{firstThreeLines(item.content)}}
</div>
</div>
{% endfor %}
<style>
.post .contents { display: hidden; }
{% for item in /data/blog %}
#{{item.id}}-check:checked~#{{item.id}} .contents { display: block; }
{% endfor %}
</style>
Example markdown file in /data/blog
:
---
title: born to die
classes: hidden
date: 2053-09-02
---
# world is a fuck
blergh
A template with a %foreach
frontmatter property will make it output many html files, and can be used to create the individual blog post pages:
---
%foreach: /data/blog
%output: /html/blog/{{page.id}}.html
---
<!doctype html>
<head>
...
<title>{{page.title}}</title>
</head>
...
<div id="{{page.id}}" class="post {{page.classes}}">
<h1 class="title">{{page.title}}</h1>
<div class="contents">
{{page.content}}
</div>
</div>
Instead of using the %foreach
frontmatter property in the post template, it's possible to 'export' the items in a for loop:
{% for item in /data/blog %}
{% export item as /html/_post.html to /html/blog/{{item.id}}.html %}
{% endfor %}
<div class="container">
<div id="sitelog" class="section">
{% import /data/sitelog/log.md %}
</div>
<div id="todo" class="section">
{% import /data/sitelog/todo.md %}
</div>
<div id="tools" class="section">
{% import /data/sitelog/tools.md %}
</div>
</div>
You can import a file as an item, then do other logic like so:
<div class="container">
{% import /data/post.md as post %}
{{css(#my-id > h1,post)}}
</div>
In _index.html
, put
<div id="blog-index">
{%%
paginate = 5
offset = 0 if 'page' not in locals() else page.offset
data = Iterators.get_data('/data/blog')[::-1]
echo <div style="background: wheat;">
echo <p>Showing post {offset+1}-{min(offset+paginate, len(data))} of {len(data)}</p>
if offset > 0:
echo <a href="index{offset//paginate-1 if offset > paginate else ""}.html">Previous</a>
if offset + paginate < len(data):
echo <a href="index{offset//paginate+1}.html">Next</a>
echo </div><br>
for i in range(offset, min(offset + paginate, len(data))):
echo <a href="{data[i].id}.html"><h2 style="background: wheat;">{data[i].title}</h1></a>
if offset + paginate < len(data):
do_export(page_item={'offset': offset + paginate}, template='_index.html', out=f'index{offset//paginate + 1}.html')
%%}
</div>
Here we create paginated index files to show a maximum of 5 posts per page. In the first block, we define constants and get the list of (reversed) posts from the data folder. In the second block, we add links to the previous and next index pages, provided it's not already the first or last page. In the third block, we add links to the posts until we reach the pagination limit. Finally, we create the next page with the same _index.html
template (recursively) by calling the do_export
function, which is the function responsible for the {% export %} command.
Note that echo string
gets converted to echo(f'string')
, this is the only preprocessing that is done before the code is executed.
convertbox.py
50 years of software development have all culminated in the single greatest application ever written.
When run, convertbox.py shows a small window where one can drag and drop files, which will then be processed by a command defined in the input box at the bottom. For example, if you want a quick png to jpeg converter, you put magick convert -quality 80 {path} {pathnoext}.jpg
in the input box and just DRAG AND DROP image files into the window.
I am a fucking genius.
playlist-convert.py
Input:
An m3u playlist file path.
Output:
- If ffmpeg is available: mp3 files with embedded images (quickly), and a m3u playlist containing song details.
- If ffmpeg is unavailable: An error message.