The Problematic Nextcloud Gallery

When I initially set up my personal Nextcloud server I was quite surprised at how unreliable it was. At seemingly random times the server would become slow or even grind to a complete halt. At first I thought my Droplet wasn’t powerful enough, had too little RAM to support quite meaty PHP server or that there were some other I/O-related bottlenecks. After a short, but nevertheless frustrating while, I found out that the slow downs occurred whenever I opened the Nextcloud gallery. Regardless of the client, be it mobile or the web interface, a single press of the gallery button would kill my Nextcloud instance. I did some digging and found a plethora of posts lamenting the poor gallery performance, not only of their own but also on professionally hosted Nextcloud installations.

High CPU utilization after a couple attempts at opening the gallery

The problem

A couple blogs and forum posts later the real culprit revealed itself to me: it turns out that Nextcloud’s thumbnail generation is at fault. More precisely, the problem is that Nextcloud generates thumbnails lazily, i.e. when an image is viewed rather than when it is uploaded. The result is that as soon as you open the gallery Nextcloud starts working… continues working… does some more work… and if you’re one of those poor souls whose gallery consists of more than two images, never stops working (until the whole server eventually stops working 😉).

Based on this observation there is reason to assume that thumbnail generation is being done sequentially even though this is one of the cases where you absolutely want to parallelize your workload. The fact that Nextcloud is based on PHP, an interpreted language, sure doesn’t help either. So I think it is safe to say that Nextcloud’s way of handling images is less-than-ideal.

How to fix it

So, we got the problem identified, now how do we fix it? I found a (by now deleted) blog post which described a reasonably pragmatic solution which I ended up applying, albeit with a few modifications. Here’s how it goes:

1. Install the Preview Generator plugin. The plugin does exactly what its name implies: it generates previews for your images. In addition to the default preview generation, however, it can be triggered manually via Nextcloud’s occ tool.

   php -f /var/www/nextcloud/occ preview:pre-generate`
   

2. Next, you want to automate the preview generation. Set up a cron job that runs the command from the previous step in regular intervals. The blog post suggests to run the job hourly but I reckon an hour window is more than enough time for a user to screw up your Nextcloud instance, i.e. to upload a bunch of images and visit the gallery (How dare they!). So in my case, I went with a one minute interval instead to spread the load more evenly. The occ tool makes sure that only one instance of the script may run at a time so it is ensured that your server will not be flooded with a bunch of concurrently running preview generation jobs. In addition I increased the processes niceness to make sure it doesn’t steal too much CPU time from the main Nextcloud server (which is assigned the default niceness of 0).
   Here’s the cron job definition that I ended up using:

   */1 * * * * nice 5 php -f /var/www/nextcloud/occ preview:pre-generate > /dev/null 2>&1
   

Conclusion

So there you have it. From now on, your thumbnails will be generated as soon as your images are uploaded (nearly), rather than when they’re viewed which will greatly improve your Nextcloud instance’s performance. Of course this is not a fix for Nextcloud’s suboptimal image handling but it is a reasonable workaround.

Bonus: Container Setup

If your Nextcloud server is running in a container you might be wondering how you would run a cronjob alongside the main Nextcloud process. If you’re like me your first instinct might be to build your own custom Docker image with crontab baked in. But before you do that, STOP! As it turns out the default Nextcloud Docker image already has crontab built in. The executable is located at /cron.sh in the container.

Equipped with this knowledge you can run a modified CMD that starts cron in the background and then runs the original entrypoint script. For more convenience you could also run supervisord, which is not part of the base image so you would have to build your own.

The only thing left to do is to mount your crontab definition into your container’s /var/spool/cron/crontabs/www-data directory. Be sure to include the default Nextcloud crontab in addition to the new preview generation crontab:

*/5 * * * *   php -f /var/www/html/cron.php
*/1 * * * *   nice 5 php -f /var/www/html/occ preview:pre-generate > /dev/null 2>&1