1. Calibre replacement considerations
  2. Summary
  3. Problems with Calibre
  4. What does Calibre do anyways

Summary

TL;DR: I'm considering replacing those various Calibre compnents with...

My biggest blocker that don't really have good alternatives are:

See below why and a deeper discussion on all the features.

Problems with Calibre

Calibre is an amazing software: it allows users to manage ebooks on your desktop and a multitude of ebook readers. It's used by Linux geeks as well as Windows power-users and vastly surpasses any native app shipped by ebook manufacturers. I know almost exactly zero people that have an ebook reader that do not use Calibre.

However, it has had many problems over the years:

Update: a previous version of that post claimed that all of Calibre had been removed from Debian. This was inaccurate, as the Debian Calibre maintainer pointed out. What happened was Calibre 4.0 was uploaded to Debian unstable, then broke because of missing Python 2 dependencies, and an older version (3.48) was uploaded in its place. So Calibre will stay around in Debian for the foreseeable future, hopefully, but the current latest version (4.0) cannot get in because it depends on older Python 2 libraries.

The latest issue (Python 3) is the last straw, for me. While Calibe is an awesome piece of software, I can't help but think it's doing too much, and the wrong way. It's one of those tools that looks amazing on the surface, but when you look underneath, it's a monster that is impossible to maintain, a liability that is just bound to cause more problems in the future.

What does Calibre do anyways

So let's say I wanted to get rid of Calibre, what would that mean exactly? What do I actually use Calibre for anyways?

Calibre is...

Note that I might have forgotten functionality in Calibre in the above list: I'm only listing the things I have used or am using on a regular basis. For example, you can have a USB stick with Calibre on it to carry the actual software, along with the book library, around on different computers, but I never used that feature.

So there you go. It's a colossal task! And while it's great that Calibre does all those things, I can't help but think that it would be better if Calibre was split up in multiple components, each maintained separately. I would love to use only the document converter, for example. It's possible to do that on the commandline, but it still means I have the entire Calibre package installed.

Maybe a simple solution, from Debian's point of view, would be to split the package into multiple components, with the GUI and web servers packaged separately from the commandline converter. This way I would be able to install only the parts of Calibre I need and have limited exposure to other security issues. It would also make it easier to run Calibre headless, in a virtual machine or remote server for extra isoluation, for example.

Update: this post generated some activity on Mastodon, follow the conversation here or on your favorite Mastodon instance.

comment 1

Calibre is a complex piece of machinery, and it's therefore buggy.

So you're going to replace it with 6 or 7 pieces of machinery, some of them complex, some of them buggy, some of them now that well maintained.

Comment by Carlos
maybe

maybe you're right. maybe the result will be too complex and buggy and will not be useful. but the reality of the problem now is that Calibre 4.0 is not going to make it to Debian until it's ported to Python 3. And even if it does, I have enough serious concerns about the security of Calibre to never want to use it again. It would need to go through a full audit, with a significant overhaul of its design and architecture, for me to ever trust it again.

I strongly doubt that the tools I have enumerated in my review will be as buggy and insecure as Calibre. Maybe I'll be proven wrong, but so far, many of those tools have proven to be well maintained and perform extremely well.

I feel it's definitely worth a try.

(I am also unsure as to which package you're refering to when you say "now that well maintained"...)

Comment by anarcat
already used tools

Another thing to understand about this list of tools is that I already use a lot of those. I already:

  • use Sphinx and Sigil to work with eBooks
  • manage and synchronize my library across multiple devices with git-annex
  • read PDFs with Evince (which Atril is derived from)
  • read RSS feeds with another feed reader

It's not like I'm hell-bent on causing the Calibre people pain here. :) It's just that I found too many problems with the software, and now it's possibly going away so I'm looking at alternatives. I thought I could share this with the community, if you don't like the answers, you're free to talk a walk elsewhere and keep on happily using Calibre, blissfully unaware of any problems I might be ranting about here. :p

Comment by anarcat
comment 4

For okular on Debian, the epub plugin is in the okular-extra-backends package. Can't tell you if it works fine (I don't use ebooks) but I can open epubs.

Comment by tonton
git-annex metadata

I've often thought that one of these days I should write a script that would parse through Calibre's metadata.opf file for each book and add it to the git-annex metadata of the ebook file. This would allow you to do all sorts of browsing, filtering and searching with git-annex's own powerful tooling, which is probably better than anything we could do today with Calibre itself (assuming you don't need a GUI).

But the problem is getting the metadata in the first place which, as you pointed out, is one of the things that Calibre does pretty well. For my video collection, I wrote metamovie to pull down metadata of movies and television shows from IMDB and store it as git-annex metadata. I suspect that it wouldn't be too difficult to hack together a similar script, using something like Google Books or Goodreads as the source. The difficult part is probably the search. With movies there is a single canonical entry, but with books you may have dozen of editions. Choosing the right one (or the one with the richest data, even if it doesn't exactly match your edition) could be a bit tricky.

Comment by Pig Monkey
Another perspective

Hi,

Thanks for writing this. So I have a Kindle, which doesn't support ePub at all. I have used Calibre at times over the years; the organization it imposes over a collection is very good, but the formatting changes in makes and the things it does during "synchronization" are not. I have mostly stopped using it because it is too cumbersome, though I do invoke ebook-viewer manually from time to time. Critically, though, Calibre does provide mobi-to-epub conversion and better support for the newer formats Amazon keeps burdening us with than most other tools. So if I were to switch to another tool, I'd have to start with Calibre and the other tool would be useful only for reading things on a PC screen (a thing I only do occasionally with ePub-type books) and not at all useful for reading things on my Kindle (a thing I do often).

This is not a defense of Calibre; I know of its quality problems first-hand, but just to say that anything that is ePub-centric is totally unhelpful to a large segment of people.

Comment by John Goerzen
fbreader has a collection browser & device sync'ing

Thanks for the info on Calibre; it seemed overengineered to me and you've just confirmed that. That said, I haven't used it since discovering fbreader.

fbreader has a fine collection browser, although on mobile you have to pay a small amount to get the version of the app which shows cover images (I did this only to support the developer). If you're willing to use Google Drive it also does device synchronization; you upload at https://books.fbreader.org and then can download to any device. It also reads several formats (epub, mobi, fb2, CBR, PDF...), so no need for conversion. My main complaint is that it doesn't reflow text in PDFs, which is annoying when reading on the small screen on my phone, but that feature's on the short-term roadmap.

I know this app is less useful to people who want to use an actual ereader and/or don't want to store their books with Google, but for others it's worth considering.

Comment by mvc
many comments

Hi everyone! Thanks for the many comments! I'll respond to the last few all together...

toton re okular: thanks! that's what i was looking for, and I confirm it works. I updated the article accordingly

monkey re git-annex: git-annex might provide the faceted search functionality that Calibre provides but, to be honest, my luck with git-annex metadata has been hit and miss. It kind of works, but for large collections, it's slow as it needs to rebuild the worktree every time you "search" something. And the user experience is really not that great: I often get confused between git branching commands and git-annex metadata commands, so much so that I quickly have no idea what's going on anymore...

john re Kindles: honestly, I don't consider Kindles as part of my workflow. I completely boycott Amazon as a company for various reasons aside from the ebook problems, but specifically their attitude regarding contents on their devices has been problematic, to say the least. I understand it was to respond to a copyright litigation, but they did delete Orwell's 1984 and Animal Farm from every Kindle device out there. They are also very aggressive in creating hurdles for us to work with their devices, as you aptly described.

So while they are shiny and enticing, hardware-wise, I don't consider them as part of my use-case model. So yes, I'm digging in my heels and I focus on standard contents: the hardware providers that do not follow that model shouldn't be encouraged with our wallets. I know it's not very helpful to the large number of Amazon users, but then again, I don't feel like going out of my way to help you people either. ;)

(And, generally, people shouldn't feel the need to defend Calibre or apologize for defending it. Again, I have used Calibre for almost a decade (probably around 7 years now, according to the first bug report), and I feel grateful for the services it has given (and still is!) giving me. So I didn't mean to write this as an attack on Calibre, but mostly as an exploration of how parts (or all) of it could be replace, if that's needed for people. It's maybe unfortunate that I started with a rather critical section, but I felt it was important to explain why I was considering this in the first place, And, I have to be honest, I kind of had a chip on my shoulder with Calibre for a while, having dealt with it through the Debian LTS security work and numerous bug reports.)

mvc re fbreader: so I've tried fbreader again and again, and I can never get used to it. It's installed on my phone, as a PDF reader, and I also have it on this laptop. Neither works very well. The phone version has this weird "bookshelf" browser that looks like a an wooden bookshelf, which is not very useful but does looks pretty. Similarly, the user interface on the desktop is ... really confusing to me. There a bunch of small buttons on top without any labels... Kind of hard to use.

The desktop version of fbreader here taks over 30 seconds to start. When I finally managed to find how to tell it where my book collection is, it crashed after a few seconds of walking the tree, with an epic segfault (Debian bug #941886):

oct 06 23:17:13 angela kernel: FBReader[16757] segfault at 8 ip 00007fb8e62800a2 sp 00007ffd84099070 error 4 in libzlcore.so.0.12.10[7fb8e626f000+54000] 
oct 06 23:17:13 angela kernel: Code: 8b 7c 24 30 48 8d 43 30 48 39 c7 74 05 e8 e6 06 ff ff 48 8b 7c 24 10 48 83 c3 10 48 39 df 74 05 e8 d3 06 ff ff 48 8b 44 24 08 <48> 8b 78 08 e8 05 f0 fe ff 48 89 ee 48 89 c7 e8 aa 01 ff ff 48 8b 

I was able to workaround the problem and I do have a library browser now, so that's pretty cool. I also like that there's an easy way to add books in there...

It also looks like fbreader is severely out of date in Debian (Debian bug #765039): it has been shipping version 0.12 forever now, while upstream is at 0.99. Which version are you using? Even then, that version is over 3 years old now, and it seems fbreader turned proprietary in 2015...

As for PDF reflows, I have found this generally never works. I have never found PDFs to be reflowable, that's just basically an impossible problem to solve, especially with LaTeX-like, scientific, two-column articles. The best, for this, is to just read the PDF on a bigger screen, and for that those bigger e-readers are just great, albeit expensive.

Comment by anarcat
Replace vs Fix

This appears to be an awful lot of effort to migrate from an open-source project. Perhaps that effort could be applied to helping out with the migration to Python3? They have a working Beta now: https://github.com/kovidgoyal/calibre/pull/870

You don't have to be a Python developer. It would be helpful just having some willing beta-testers provide some feedback.

Either way, thanks for providing a roadmap of comparable software. I personally don't want to switch from Calibre, but seeing the alternatives is always helpful for finding ways to improve.

  • Jim
Comment by Jim Sorenson
comment 11

I have only used the latest stable fbreader Android app, which is proprietary but still actively maintained, and for me "just works". When the project went closed source in 2015 a free software fork targeting Android was created, but I've never tried it: https://gitlab.com/axet/android-book-reader

The free app's bookshelf view was a little annoying, but as I said I was happy to give the developer a small sum of money for the "premium" version with a better collection viewer.

I'm not surprised that PDF reflowing would be impossible in the general case for a multicolumn document, but I'm hoping they'll come up with something workable for some of the lengthy books I want to read that were only available in PDF format from the local library. At least I can borrow the physical books if needed!

Comment by mvc
comment 11

This appears to be an awful lot of effort to migrate from an open-source project. Perhaps that effort could be applied to helping out with the migration to Python3? They have a working Beta now: https://github.com/kovidgoyal/calibre/pull/870

You don't have to be a Python developer. It would be helpful just having some willing beta-testers provide some feedback.

I actually am a Python developer. If Python 3 was the only problem with Calibre, I would totally and enthousiastically go that route. But as I explained in the blog post, there are other problems here.

Also, there's only so much time in the day. I am already involved in another major Python 3 conversion in a program we salvaged from being abandoned by its maintainer (linkchecker) and it's basically taking all of my free time at that level right now. I have my own programs I need to port to Python 3. So, sorry, but I don't have (free or paid, actually) time to offer to the Calibre community right now. But you should know that I did contribute quite a few times in the community (bug reports, security backports and patches). My experience then wasn't as great as I would have liked and I prefer not to get involved anymore.

Either way, thanks for providing a roadmap of comparable software. I personally don't want to switch from Calibre, but seeing the alternatives is always helpful for finding ways to improve.

Thanks! That's the spirit with which I'm writing this. I might end up continuing to use Calibre myself too!

Comment by anarcat
Calibre replacements

Yeah, I noticed some problems with Calibre as well, in 2013.

https://seegras.discordia.ch/Blog/life-with-calibre/

I've written some software then, mainly to speed up extraction of epub-metatags by a factor of 100(!), but it's grown into a veritable zoo of scripts. You can find them on my site. Mainly:

  • https://github.com/Seegras/epub-meta
  • https://github.com/Seegras/metatags

... might be helpful.

In any case, your posting helped me too, if for not other reason than I know now that atril can read epubs; but hopefully this calibre issue leads to more epub-related software packaged for debian.

lucidor, btw., was nice, but it's ancient. Not worth porting anymore.

BTW: you're managing your books in git-annex? How does that work exactly? I'm interested to read another blog-posting about that ;).

Comment by Peter Keel
git-annex
Re. git-annex, nothing special. I had some hacks to make it ignore the .db file, because Calibre would be really upset if it can't open (ie. write) to the database. But now with "unlocked mode", it's not really a problem anymore.
Comment by anarcat
Or an epub to HTML converter plus lynx…

I wrote this small shellscript for myself…

http://www.mirbsd.org/cvs.cgi/contrib/hosted/tg/unepub?rev=HEAD

… which unzips an EPUB file and converts the toc.nxc into an index.htm which I can then use with lynx (the standard text mode webbrowser).

Comment by mirabilos
Created . Edited .