November 2017 report: LTS, standard disclosure, Monkeysphere in Python, flash fraud and Goodbye Drupal

  1. blog
  1. Debian Long Term Support (LTS)
    1. Exiv
    2. Libreoffice
    3. OptiPNG
    4. Miscellaneous
  2. Other free software work
    1. feed2exec
    2. Towards standard security disclosure guidelines
    3. Git-Mediawiki
    4. Rewriting Monkeysphere in Python?
    5. Goodbye, Drupal
    6. Fixing fraudulent memory cards
    7. Miscellaneous

Debian Long Term Support (LTS)

This is my monthly Debian LTS report. I didn't do as much as I wanted this month so a bunch of hours are reported to next month. I got frustrated by two difficult packages: exiv2 and libreoffice.

Exiv

For Exiv2 I first reported the issues upstream as requested in the original CVE assignment. Then I went to see if I could reproduce the issue. Valgrind didn't find anything, so I went on to test the new ASAN instructions that tell us how to build for ASAN in LTS. Turns out that I couldn't make that work either. Fortunately, Roberto was able to build the package properly and confirmed the wheezy version as non-vulnerable, so I marked the three CVEs as not-affected and moved on.

Libreoffice

Next up was LibreOffice. I started backporting the patches to wheezy which was a little difficult because any error in the backport takes hours to find because LibreOffice is so big. The monster takes about 4 hours to build on my i3-6100U processor - I can't imagine how long that would take on a slower machine. Still, I managed to get patches that mostly builds. I say mostly because while most of the code builds, the tests fail to build. Not only do they fail to build, but they even segfault the linker. At that point, I had already spent too many hours working on this frustrating loop of "work/build-wait/crash" that I gave up.

I also worked on reproducing a supposed regression associated with the last security update. Somehow, I couldn't reproduce either - the description of the regression was very limited and all suggested approaches failed to trigger the problems described.

OptiPNG

Finally, a little candy: an easy backport of a simple 2-line patch for a simple program, OptiPNG that, ironically, had a vulnerability (CVE-2017-16938) in GIF parsing. I could do hundreds of those breezy updates, they are fun and simple, and easy to test. This resulted in the trivial DLA-1196-1.

Miscellaneous

LibreOffice stretched the limits of my development environment. I had to figure out how to deal with out of space conditions in the build tree (/build) something that is really not obvious in sbuild. I ended up documenting that in a new troubleshooting section in the wiki.

Other free software work

feed2exec

I pushed forward with the development of my programmable feed reader, feed2exec. Last month I mentioned I released the 0.6.0 beta: since then 4 more releases were published, and we are now at the 0.10.0 beta. This added a bunch new features:

The documentation deserves special mention. If you compare between version 0.6 and the latest version you can see 4 new sections:

All those changes were backported in the ecdysis template documentation and I hope to backport them back into my other projects eventually. As part of my documentation work, I also drifted into the Sphinx project itself and submitted a patch to make manpage references clickable as well.

I now use feed2exec to archive new posts on my website to the internet archive, which means I have an ad-hoc offsite backup of all content I post online. I think that's pretty neat. I also leverage the Linkchecker program to look for dead links in new articles published on the site. This is possible thanks to a Ikiwiki-specific filter to extract links to changed files from the Recent Changes RSS feed.

I'm considering making the parse step of the program pluggable. This is an idea I had in mind for a while, but it didn't make sense until recently. I described the project and someone said "oh that's like IFTTT", a tool I wasn't really aware of, which connects various "feeds" (Twitter, Facebook, RSS) to each other, using triggers. The key concept here is that feed2exec could be made to read from Twitter or other feeds, like IFTTT and not just write to them. This could allow users to bridge social networks by writing only to a single one and broadcasting to the other ones.

Unfortunately, this means adding a lot of interface code and I do not have a strong use case for this just yet. Furthermore, it may mean switching from a "cron" design to a more interactive, interrupt-driven design that would run continuously and wake up on certain triggers.

Maybe that could come in a 2.0 release. For now, I'll see to it that the current codebase is solid. I'll release a 0.11 release candidate shortly, which has seen a major refactoring since 0.10. I again welcome beta testers and users to report their issues. I am happy to report I got and fixed my first bug report on this project this month.

Towards standard security disclosure guidelines

When reading the excellent State of Opensource Security report, some metrics caught my eye:

In other words, having a public disclosure policy more than triples your chances of being notified of a security vulnerability. I was also surprised to find that 4 out of 5 projects do not have such a page. Then I realized that none of my projects had such a page, so I decided to fix that and fix my documentation templates (the infamously named ecdysis project) to specifically include a section on security issues.

I found that the HackerOne disclosure guidelines were pretty good, except they require having a bounty for disclosure. I understand it's part of their business model, but I have no such money to give out - in fact, I don't even pay myself for the work of developing the project, so I don't quite see why I would pay for disclosures.

I also found that many projects include OpenPGP key fingerprints in their contact information. I find that's a little silly: project documentation is no place to offer OpenPGP key discovery. If security researchers cannot find and verify OpenPGP key fingerprints, I would be worried about their capabilities. Adding a fingerprint or key material is just bound to create outdated documentation when maintainers rotate. Instead, I encourage people to use proper key discovery mechanism like the Web of trust, WKD or obviously TOFU which is basically what publishing a fingerprint does anyways.

Git-Mediawiki

After being granted access to the Git-Mediawiki project last month, I got to work. I fought hard with both Travis and Git, and Perl, and MediaWiki, to add continuous integration in the repository. It turns out the MediaWiki remote doesn't support newer versions of MediaWiki because the authentication system changed radically. Even though there is supposed to be backwards-compatibility, it doesn't seem to really work in my cases, which means that any test case that requires logging into the wiki fails. This also required using an external test suite instead of the one provided by Git, which insists on building Git before being used at all. I ended up using the Sharness project and submitted a few tests that were missing.

I also:

I also opened a discussion regarding the versioning scheme and started tracking issues that would be part of the next release. I encourage people to get involved in this project if they are interested: the door is wide open for contributors, of course.

Rewriting Monkeysphere in Python?

After being told one too many times that Monkeysphere doesn't support elliptic curve algorithms, I finally documented the issue and looked into solutions. It turns out that a key part of Monkeysphere is a Perl program written using a library that doesn't support ECC itself, which makes this problem very hard to solve. So I looked into the PGPy project to see if it could be useful and it turns out that ECC support already exists - the only problem is the specific curve GnuPG uses, ed25519, was missing. The author fixed that but the fix requires a development version of OpenSSL, because even the 1.1 release doesn't support that curve.

I nevertheless went ahead to see how hard it would be to re-implement that key component of Monkeysphere in Python, and ended up with monkeypy, an analysis of the Monkeysphere design and first prototype of a Python-based OpenPGP / SSH conversion tool. This lead to a flurry of issues on the PGPy project to fix UTF-8 support, easier revocation checks and a few more. I also reviewed 9 pull requests that were pending in the repository. A key piece missing is keyserver interaction and I made a first prototype of this as well.

It was interesting to review Monkeysphere's design. I hope to write more about this in the future, especially about how it could be improved, and find the time to do this re-implementation which could mean wider adoption of this excellent project.

Goodbye, Drupal

I finally abandoned all my Drupal projects, except the Aegir-related ones, which I somewhat still feel attached to, even though I do not do any Drupal development at all anymore. I do not use Drupal anywhere (unless as a visitor on some websites maybe?), I do not manage Drupal or Aegir sites anymore, and in fact, I have completely lost interested in writing anything remotely close to PHP.

So it was time to go. I have been involved with Drupal for a long time: I registered on Drupal.org in March 2001, almost 17 years ago. I was active for about 12 years on the site: I made my first post in 2003 which showed I was already "hacking core" which back then was the way to go. My first commit, to the mailman_api project, was also in 2003, and I kept contributing until my last commit in 2015, on the Aegir project of course.

That is probably the longest time I have spent on any software project, with the notable exception of the Debian project. I had a lot of fun working with Drupal: I met amazing people, traveled all over the world and made powerful websites that shook the world. Unfortunately, the Drupal project has decided to go in a direction I cannot support: Drupal 8 is a monster that is beyond anything needed for me, my friends or the organizations I support. It may be very well for the startups and corporations, to create large sites, but it seems it completely lost touch with its roots. Drupal used to be a simple tool for "community plumbing" (ref), it is now a behemoth to "Launch, manage, and scale ambitious digital experiences—with the flexibility to build great websites or push beyond the browser" (ref).

When I heard a friend had his small artist website made in Drupal, my first questions were "which version?" and "who takes care of the upgrades?" Those questions still don't seem to be resolved in the community: a whole part of the project is still stuck in the old Drupal 7 world, years after the first D8 alpha releases. Drupal 8 doesn't address the fundamental issue of upgradability and cost: Drupal sites are bigger than they ever were. The Drupal community seems to be following a very different way of thinking (notice the Amazon Echo device there) than what I hope the future of the Internet will be.

Goodbye, Drupal - it's been a great decade, but I'm already gone and I'm not missing you at all.

Fixing fraudulent memory cards

I finally got around testing the interesting f3 project with a real use case, which allowed me to update the stressant documentation with actual real-world results. A friend told me the memory card he put in his phone had trouble with some pictures he was taking with the phone: they would show up as "gray" when he would open them, yet the thumbnail looked good.

It turns out his memory card wasn't really 32GB as advertised, but really 16GB. Any write sent to the card above that 16GB barrier was just discarded and reads were returned as garbage. But thanks to the f3 project it was possible to fix that card so he could use it normally.

In passing, I worked on the f3 project to merge the website with the main documentation, again using the Sphinx project to generate more complete documentation. The old website is still up, but hopefully a redirection will be installed soon.

Miscellaneous

I spent only 30% of this month on paid work.

RSS
f3

Maybe add a note to:

make experimental

for:

  • f3probe
  • f3fix
Comment by Anonyme —
D8

Drupal 8 is clearly not oriented towards giving a publishing tool to the comunity anymore. All development since D8 is meant to make big projects possible.

Also D8 breaks its own API on minor releases.

So they've only exacerberated what was already problematic: maintaining a website that uses D8 costs way more money than did already previous versions of Drupal.

So there's no surprise seeing that most of its comunity is not following anymore. Most ppl using Drupal used to be from backgrounds different than big corporations.

Comment by LeLutin —
Comments on this page are closed.
Created . Edited .