Welcome to johnbintz.com

I’m now maintaining the Sandstorm App Store version of DokuWiki, a nice, lightweight wiki webapp.

I use DokuWiki along with Wekan to organize my comic and animation projects. Both The Wizard and The Metalsmith and Rabbit with 1000 Repos have a combo of DokuWiki for scripts, marketing details, links/reference material, and other long-form text content, and Wekan is my big ol' kanban board for managing work.

Since the original DokiWiki was a few years out of date, and a lot has changed both in DokuWiki and the PHP world in general, the app’s going to be updated slowly to the latest revision, so those folks with plugins installed that may fail on newer DokuWikis have time to get that resolved before new versions come along. Check the GitHub Repo for more information on that process, and if you have PRs/questions/etc.

I want to take over a Sandstorm app! How do?

So you have a favorite app you want to start maintaining, and it might even be one of mine if I decide to stop maintaining Hugo or DokuWiki. How do you take it over? Well, here’s the process I went through with taking over Hugo and DokuWiki. This is current as of July 2020.

1. You’ll need a Keybase account. Depending on what Zoom does with Keybase, this info may become out of date.
2. Install Vagrant and vagrant-spk.
3. Get the project running locally, in its current state and at the version matching what’s in the App Store. Depending on the quality of the project’s repository and the project itself, this may be very easy or very hard. This might also require a lot of internet searching, lots of machine provisioning and reprovisioning using Vagrant, learning how to log debug output to stdout and stderr from whatever services encompass the app, and poking around a running grain using vagrant-spk enter-grain. Unless something is truly broken, this is not the time to fix up minor annoyances.
4. Export this unupdated app to a .pkg file via vagrant-spk pack. You’ll have to do the GPG and Keybase dance to be able to sign the package, so create a GPG key if you don’t already have one. Commit and push a new branch with this set of modifications so you can roll back to this state easily.
5. Upgrade the app. Depending on the application, this may involve a small upgrade at first, or may involve going straight to the latest version. I went to the latest version with Hugo and added a note about breaking changes in the admin area of the app as well as on the app’s app store page. With DokuWiki, I planned a set of smaller upgrades, taking a few weeks to get up to the latest version, since DokuWiki is a much more complicated piece of software to upgrade than Hugo.
6. Get the updated app running correctly, and export it to a different .pkg file via vagrant-spk pack.
7. Create a fake Sandstorm project that doesn’t have an app associated with it using vagrant-spk init in an empty folder. Install the unupdated app via direct upload of the .pkg file into this fake project and make sure it works correctly, spawning whatever grains you need to for testing. You do not need to use vagrant-spk dev for this part. Starting the VM gives you a sandboxed Sandstorm to mess with without doing anything else.
8. Upload the updated app .pkg and upgrade your test grains in this VM. Make sure a grain upgrade works correctly.
9. If the original repo has a message about looking for a new maintainer, try asking there if they want to turn it over to you. Otherwise, head to the sandstorm-dev mailing list and state your intent to take over maintenance.
10. You’ll eventually be given a keyfile to cat onto your Sandstorm keyring. Do just that: cat < provided-key >> ~/.sandstorm/sandstorm-keyring. Ensure the app ID for the original app shows up with vagrant-spk listkeys. Back up your keyring before doing the cat just in case!
11. Publish your app to the app store and wait for feedback. Depending on the level of changes, it can be a while to get feedback and to get the updated app to the quality level it needs to be at.
12. After your app has made it onto the store and you’ve successfully done a release or two, consider automating the upgrade process as much as you can, where you can, to reduce the friction of putting out new releases. I do this with hugo-sandstorm and dokuwiki-sandstorm.
13. Wherever you’re hosting the repo, ensure you can get feedback from users, and, more importantly, code from contributors.
14. Once you have a few releases out, and can maintain a release schedule, then worry about fixing minor annoyances or edge case bugs. Remember, you’re spending your free time on this, so choose wisely what you work on, and ask for help when possible!
15. If you ever decide to give up maintenance, let sandstorm-dev know, put a notice on the README for the repository, and (optionally) make the repo read-only so that no new issues can be posted. If someone else wants to take it over, they can fork it and repeat this very process!

Some of these instructions are specific for Linux. All of them are the process I need to remember to walk through to get the pen working correctly. I also have 8 rechargeable AAAA batteries and two Pen Pros.

• Does the docked pen work? The docked pen is too uncomfortable to use for long-term drawing and it’s only for testing at this point.
  • If no, check dmesg for kfifo events.
    • If kfifo events, sudo rmmod wacom; sudo modprobe wacom
• Does the Pen Pro work on another ThinkPad? I have two Yogas: a personal one and one for work.
  • If yes, try restarting the one the pen does not work on.
• Try all batteries in the first Pen Pro.
  • Remove the existing battery.
  • Put the new battery in but don’t screw the cap on all the way.
  • Hold the pen tip up to the screen.
  • Slowly screw the top on, then unscrew it.
    • If you see the mouse moving, congrats, you got it.
    • If not try a new battery.
• Try all batteries in the other Pen Pro.
• If no batteries work in any Pen Pro, recharge all the batteries and try again.
• If they still don’t work, clean the screw threads in the caps of the pens with alcohol and try again.
• If they still don’t work, add conductive thread to the cap to increase the contact area, though I don’t know how much that helps now that I’ve had that setup for about a year now.

I really hate needing styluses that take or contain batteries. Older Wacoms (and Samsung S-Pens) were fine without them. :/ And I hate the idea of buying a bunch of AAAA alkaline batteries and throwing them away just to draw.

With the animation I’m working on I want some help breaking down the lip movements that Bamboo is going to have to make while talking. I saw that Synfig has support for loading Papagayo lip sync files so I took a look at the 2.0, C++ version of the project and saw it was quite abandoned.

I decided to incorporate a bunch of the great fixes to the software over the years, as well as a tweak of my own, to the all-the-fixes branch on my forked repo. And, to make sure it’s working as intended, I created a lip sync from my animatic audio in the forked version:

…and loaded it into Synfig:

I’m not looking to become a hardcore maintainer of Papagayo. I just want it working well enough for my animation work. If you want to help keep it going, pull requests are best!

Update: I found the Python-based 1.x fork called Papagayo-NG which seems better maintained and has AppImages. I may try this one out as well. The project is also more active than the original one.

So you want to do hand-drawn animation on Linux and/or entirely with Free Open Source Software, and that animation involves lip syncing? Well, here’s some notes as I work on my first decent sized animation in Synfig/Krita/Kdenlive and I get my pipeline ready to go for smooth production work on future animation.

• Get the latest development version if you’re on Ubuntu, the version (as of 2020-06-03) in the Ubuntu repos has a bug where you can’t enter in 0 for the angle of a rotation, and you also can’t use Ctrl-A to select the contents of a field in the Properties of a group, and I’ll risk any other weirdness in the app to be able to enter in 0 for the angle of a rotation and to use Ctrl-A.

  • If you want better (imho) Papagayo import that doesn’t forcibly put a rest at the end of every word, grab this branch and build it. The build process for Synfig is very very nice, by the way! I wish all large apps were this smooth to build and rebuild.

• You should try out this version of Papagayo, cobbled together from various forks over the years, that is fast and supports single-frame words and has other nice quality of life improvements. Please help me get it building on other platforms!

• If you’re running a modern Ubuntu and using the Synfig AppImage, there’s probably gonna be a ton of fontconfig errors and the UI will look weird. Errors mentioning unknown element. Lots of them. This happens to me in Krita as well on another machine when run via AppImage, though this did not fix Krita on that machine. The solution is to run the AppImage and provide the system fontconfig libraries via LD_PRELOAD. The exact line I had to use is:

  LD_PRELOAD="/usr/lib/x86_64-linux-gnu/libfontconfig.so:/usr/lib/x86_64-linux-gnu/libfreetype.so" \
  Downloads/SynfigStudio-1.3.14-testing-2020.04.30-linux64-b2662.appimage
  

  Hunt down those two libraries in /usr/lib and replace the paths with what you have and the fonts will look all pretty and stuff. Or build Synfig from source and you won’t have this issue.

• Inkscape .sif export is not well supported, even for things like rectangles. Synfig also doesn’t seem to like Inkscape SVG files. This is fine, as I’m doing titling in Kdenlive and only importing hand-drawn bitmaps from Krita into Synfig.

• I was going to do the whole thing in Synfig, but I found a combination of Krita, Synfig, and Kdenlive is what you’ll probably want. I envision a much larger, more involved blog post/video series once my Papagayo fix is in for Synfig. This also means separate Synfig files for, say, camera distances from things, so you don’t need to worry as much about RAM usage in Synfig as you’re making a lot of smaller scenes (exported at HuffYUV-compressed MP$ files) and assembline them in Kdenlive.

• By default, importing a bitmap is…wrong. The dimensions are all messed up. Edit > Preferences > Editing > Imported Image > Scale to fit canvas is what you want. This means you’re importing in everything at full size, so export everything at full size from Krita. The exports are compressed PNG files, and disk space is cheap nowadays.

• It looks like you can easily build at a lower resolution, say 640x360, and then export to a larger resolution, say 2560x1440, and everything scales as it should. As long as the assets are high enough quality, of course.

• This pipeline works as expected because Synfig is storing references to imported images and not the whole image data:

  • Export rough animation frames in Krita
  • Import frames as image sequence into Synfig
  • Export cleaned up animation frames with same names in Krita
  • Reopen the Synfig file
  • The new, cleaned up frames will appear in Synfig

• Only put one voice into a Papagayo file, otherwise none of the voices will appear when imported into Synfig.

  • It seems like it’s best to split the audio files into one clip per Papagayo file per character pose. In one scene, Bamboo will have two body/face positions, so I’ll need two clips and two Papagayo files.

• The parts you need for your lipsync from Papagayo are listed on this lip sync chart with a few changes:

  • L is L, Th
  • etc is C, D, …

  Which means you’ll, at most, need art for: AI, O, E, U, L, WQ, MBP, FV, etc, rest

• The best way I found to make the mouth parts and export them in a way that makes Synfig’s handling of them less of a pain is:

  • Create a new group layer above the layer where the character’s head is located.
  • Draw each mouth, in a new layer, with the layer named after the part.
  • Use Krita Batch Exporter to export the frames, one image per mouth layer.
    • The exporter can also export group layers, with visible layers merged down, so my standard inks -> shading -> colors layer setup will work just fine.
  • Import the Papagayo file into Synfig.
    • This is actually a link! The frames are reloaded if the Papagayo file is updated. (or if you modify Synfig’s source code to process a Papagyo file differently!)
  • Import each mouth image individually, and drag the imported image into the appropriate folder in the Papagayo group object.
    • If the Papagayo file isn’t openable in the Layer docker in Synfig, put the file in a Group, then remove it from the Group. I tend to put everything imported in a Group now.
  • BOOM, mouth animation.
  • Since Synfig supports Python plugins that work by manipulting a temporary Synfig (XML) document during an import, I’ll figure out a way to import all of the mouth patterns for a Papagayo file somehow. Or, I’ll just make a Python or Ruby script to modify the SIF file and put in the links via the command line. Yay XML! :(

• Below is a Python script to run in Scripter in Krita to create layers for each mouth pattern used in a Papagayo file. You only get the layers for the mouth patters you’re actually using, which will save on drawing. Select a Group layer where the new layers should be deposited and run it. This sets up the layers I will typically use for an art piece: Pencils, Colors, Shading (set up Multiply blending mode), and Inks. This works fine in Krita 4.3.0:

  from PyQt5.QtWidgets import QFileDialog
  from krita import Krita
  
  KID = Krita.instance().activeDocument()
  
  active = KID.activeNode()
  
  file, _ = QFileDialog.getOpenFileName(None, "Create phoneme layers for Papagayo file", "", "Papagayo File (*.pgo)")
  
  # rest is the default when a frame doesn't have a specified phoneme
  phonemes = ["rest"]
  
  with open(file) as f:
      for line in f:
          line = line.rstrip()
          if line.startswith("\t\t\t\t"):
              _, phoneme = line.split(" ")
              if not(phoneme in phonemes):
                  phonemes.append(phoneme)
  
  for name in phonemes:
      layer = KID.createGroupLayer(name)
      active.addChildNode(layer, None)
  
      for artName in ["Pencils", "Colors", "Shading", "Inks"]:
          artLayer = KID.createNode(artName, "paintlayer")
  
          if artName == "Shading":
              artLayer.setBlendingMode("multiply")
  
          layer.addChildNode(artLayer, None)
  

All in all, the process was pretty decent. It took some time to figure out the Synfig way of doing things, but once I got that, the actual pipeline of Krita -> Synfig -> Kdenlive was very smooth. Expect something way more substantial describing the process in the coming months.

I host all my sites on Sandstorm using Hugo and hugo-sandstorm. This is super convenient for a couple reasons:

• I don’t have to worry about deploys for new or existing sites, a git push takes care of it.
• I don’t need to architect the git push setup, the app does that for me.
• I can easily run the blog locally for testing.
• Because of sandcats, each site magically gets a subdomain I can point at, either with a CNAME or a proxy like HAProxy or nginx.

However, there are limitations:

• No 404 pages.
• No easy way to use your own HTTPS certificates like Let’s Encrypt ones.
• I want to clamp down on subdomains so www.johnbintz.com redirects properly.

I was solving the latter 2 for a while with HAProxy, but after setting up the site for Rabbit with 1000 Repos and realizing I wanted to shift pages and sections around as the dust settles on this hit new comic, I wanted 404 pages so folks could still find what they were looking for.

Here’s the nginx config I eventually came up with. You’ll still have to put the site’s public ID as a TXT record into your domain name as indicated in your static config setup, and make sure the Host header is sent along correctly so Sandstorm can do the DNS lookup correctly:

# redirect http to https without wildcards
server {
  listen 80;

  server_name ~^.*\.johnbintz.com$;

  return 301 https://johnbintz.com$request_uri;
}

# serve subdomainless from sandstorm
server {
  listen 443 ssl;

  # let's encrypt certificates go here

  server_name ~^johnbintz.com$;

  location / {
    proxy_set_header Host johnbintz.com;
    proxy_pass https://the-sandcats-url-sandstorm-static-publishing-gives-you;
    proxy_intercept_errors on;
    error_page 404 /404/;
 }
}

Then, if you’re using Hugo, create content/404.md with the contents of your 404 page. In the event of a missing page, the user will be handed the content of this 404 page, and receive an HTTP 404 Not Found header in response, so search engines will do the right thing, and it’s way better than the blank Cannot GET /this-page-does-not-exist page you get normally with Sandstorm static publishing.