My free software activities, June 2018
It's been a while since I haven't done a report here! Since I need to do one for LTS, I figured I would also catchup you up with the work I've done in the last three months. Maybe I'll make that my new process: quarterly reports would reduce the overhead on my side with little loss on you, my precious (few? many?) readers.
Debian Long Term Support (LTS)
This is my monthly Debian LTS report.
I omitted doing a report in May because I didn't spend a significant number of hours, so this also covers a handful of hours of work in May.
May and June were strange months to work on LTS, as we made the transition between wheezy and jessie. I worked on all three LTS releases now, and I must have been absent from the last transition because I felt this one was a little confusing to go through. Maybe it's because I was on frontdesk duty during that time...
For a week or two it was unclear if we should have worked on wheezy, jessie, or both, or even how to work on either. I documented which packages needed an update from wheezy to jessie and proposed a process for the transition period. This generated a good discussion, but I am not sure we resolved the problems we had this time around in the long term. I also sent patches to the security team in the hope they would land in jessie before it turns into LTS, but most of those ended up being postponed to LTS.
Most of my work this month was spent actually working on porting the Mercurial fixes from wheezy to jessie. Technically, the patches were ported from upstream 4.3 and led to some pretty interesting results in the test suite, which fails to build from source non-reproducibly. Because I couldn't figure out how to fix this in the alloted time, I uploaded the package to my usual test location in the hope someone else picks it up. The test package fixes 6 issues (CVE-2018-1000132, CVE-2017-9462, CVE-2017-17458 and three issues without a CVE).
I also worked on cups in a similar way, sending a test package to the security team for 2 issues (CVE-2017-18190, CVE-2017-18248). Same for Dokuwiki, where I sent a patch single issue (CVE-2017-18123). Those have yet to be published, however, and I will hopefully wrap that up in July.
Because I was looking for work, I ended up doing meta-work as well. I
made a prototype that would use the embedded-code-copies file
to populate data/CVE/list with related packages as a way to address
a problem we have in LTS triage, where package that were renamed
between suites do not get correctly added to the tracker. It ended up
being rejected because the changes were too invasive, but led to
Brian May suggesting another approach, we'll see where that goes.
I've also looked at splitting up that dreaded data/CVE/list but my
results were negative: it looks like git is very efficient at
splitting things up. While a split up list might be easier on editors,
it would be a massive change and was eventually refused by the
security team.
Other free software work
With my last report dating back to February, this will naturally be a little imprecise, as three months have passed. But let's see...
LWN
I wrote eigth articles in the last three months, for an average of three monthly articles. I was aiming at an average of one or two a week, so I didn't get reach my goal. My last article about Kubecon generated a lot of feedback, probably the best I have ever received. It seems I struck a chord for a lot of people, so that certainly feels nice.
Linkchecker
Usual maintenance work, but we at last finally got access to the Linkchecker organization on GitHub, which meant a bit of reorganizing. The only bit missing now it the PyPI namespace, but that should also come soon. The code of conduct and contribution guides were finally merged after we clarified project membership. This gives us issue templates which should help us deal with the constant flow of issues that come in every day.
The biggest concern I have with the project now is the C parser and the outdated Windows executable. The latter has been removed from the website so hopefully Windows users won't report old bugs (although that means we won't gain new Windows users at all) and the former might be fixed by a port to BeautifulSoup.
Email over SSH
I did a lot of work to switch away from SMTP and IMAP to synchronise my workstation and laptops with my mailserver. Having the privilege of running my own server has its perks: I have SSH access to my mail spool, which brings the opportunity for interesting optimizations.
The first I have done is called rsendmail. Inspired by work from
Don Armstrong and David Bremner, rsendmail is a Python program I wrote
from scratch to deliver email over a pipe, securely. I do not trust
the sendmail command: its behavior can vary a lot between platforms
(e.g. allow flushing the mailqueue or printing it) and I wanted to
reduce the attack surface. It works with another program I wrote
called sshsendmail which connects to it over a pipe. It integrates
well into "dumb" MTAs like nullmailer but I also use it with the
popular Postfix as well, without problems.
The second is to switch from OfflineIMAP to Syncmaildir (SMD). The latter allows synchronization over SSH only. The migration was a little difficult but I very much like the results: SMD is faster than OfflineIMAP and works transparently in the background.
I really like to use SSH for email. I used to have my email password stored all over the place: in my Postfix config, in my email clients' memory, it was a mess. With the new configuration, things just work unattended and email feels like a solved problem, at least the synchronization aspects of it.
Emacs
As often happens, I've done some work on my Emacs configuration. I switched to a new Solarized theme, the bbatsov version which has support for a light and dark mode and generally better colors. I had problems with the cursor which are unfortunately unfixed.
I learned about and used the Emacs iPython Notebook project (EIN) and filed a feature request to replicate the "restart and run" behavior of the web interface. Otherwise it's real nice to have a decent editor to work on Python notebooks and I have used this to work on the terminal emulators series and the related source code
I have also tried to complete my conversion to Magit, a pretty nice wrapper around git for Emacs. Some of my usual git shortcuts have good replacements, but not all. For example, those are equivalent:
vc-annotate(C-x C-v g):magit-blamevc-diff(C-x C-v =):magit-diff-buffer-file
Those do not have a direct equivalent:
vc-next-action(C-x C-q, or F6):anarcat/magic-commit-buffer, see belowvc-git-grep(F8): no replacement
I wrote my own replacement for "diff and commit this file" as the following function:
(defun anarcat/magit-commit-buffer ()
"commit the changes in the current buffer on the fly
This is different than `magit-commit' because it calls `git
commit' without going through the staging area AKA index
first. This is a replacement for `vc-next-action'.
Tip: setting the git configuration parameter `commit.verbose' to
2 will show the diff in the changelog buffer for review. See
`git-config(1)' for more information.
An alternative implementation was attempted with `magit-commit':
(let ((magit-commit-ask-to-stage nil))
(magit-commit (list \"commit\" \"--\"
(file-relative-name buffer-file-name)))))
But it seems `magit-commit' asserts that we want to stage content
and will fail with: `(user-error \"Nothing staged\")'. This is
why this function calls `magit-run-git-with-editor' directly
instead."
(interactive)
(magit-run-git-with-editor (list "commit" "--" (file-relative-name buffer-file-name))))
It's not very pretty, but it works... Mostly. Sometimes the
magit-diff buffer becomes out of sync, but the --verbose output in
the commitlog buffer still works.
I've also looked at git-annex integration. The magit-annex package did not work well for me: the file listing is really too slow. So I found the git-annex.el package, but did not try it out yet.
While working on all of this, I fell in a different rabbit hole: I
found it inconvenient to "pastebin" stuff from Emacs, as it would
involve selection a region, piping to pastebinit and copy-pasting
the URL found in the *Messages* buffer. So I wrote this first
prototype:
(defun pastebinit (begin end)
"pass the region to pastebinit and add output to killring
TODO: prompt for possible pastebins (pastebinit -l) with prefix arg
Note that there's a `nopaste.el' project which already does this,
which we should use instead.
"
(interactive "r")
(message "use nopaste.el instead")
(let ((proc (make-process :filter #'pastebinit--handle
:command '("pastebinit")
:connection-type 'pipe
:buffer nil
:name "pastebinit")))
(process-send-region proc begin end)
(process-send-eof proc)))
(defun pastebinit--handle (proc string)
"handle output from pastebinit asynchronously"
(let ((url (car (split-string string))))
(kill-new url)
(message "paste uploaded and URL added to kill ring: %s" url)))
It was my first foray into aynchronous process operations in Emacs: difficult and confusing, but it mostly worked. Those who know me know what's coming next, however: I found not only one, but two libraries for pastebins in Emacs: nopaste and (after patching nopaste to add asynchronous support and customize support of course) debpaste.el. I'm not sure where that will go: there is a proposal to add nopaste in Debian that was discussed a while back and I made a detailed report there.
Monkeysign
I made a minor release of Monkeysign to cover for CVE-2018-12020 and its GPG sigspoof vulnerability. I am not sure where to take this project anymore, and I opened a discussion to possibly retire the project completely. Feedback welcome.
ikiwiki
I wrote a new ikiwiki plugin called bootstrap to fix table markup to match what the Bootstrap theme expects. This was particularly important for the previous blog post which uses tables a lot. This was surprisingly easy and might be useful to tweak other stuff in the theme.
Random stuff
- I wrote up a review of security of APT packages when compared with the TUF project, in TufDerivedImprovements
- contributed to about 20 different repositories on GitHub, too numerous to list here