Nextcloud

October 14, 2020

Nextcloud is a file, calendar, contact sharing web application (and much much more).

May First administers a Nextcloud instance shared by all members - any user created in the control panel with access granted to Nextcloud can login to this shared instance. In addition, we can also can spin up instances dedicated to specific members.

System design

There are six main components to a Nextcloud installation:

The database service: We use a Postgres server running on a dedicated virtual machine (psql003)
The web service: We run nginx on the nextcloud KVM guest (nextcloud002).
The PHP service: We run PHP FPM on the nextcloud KVM guest (nextcloud002), allowing us to run multiple instances of Nextcloud (e.g. live and staging).
The redis service (key/value database to speed things up): We run a separate redis instance on the Nextcloud KVM guest for each instance of Nextcloud, managed by separate systemd service files.
The notify_push service: We run separate notify push services on the Nextcloud KVM guest for each Nextcloud instance that runs notify_push. This service is a program written in rust and efficiently handles push requests for clients, thus reducing the CPU load on the PHP server.
The files: all files are served from the Nextcloud KVM guest.

Setting up an instance

Choose a user for the instance (which will be referred to below as $USER). This user becomes a label used throughout the process: the name of the database, name of the database user, etc, so, please use all lower case and restrict to just the letters a-z - no dashes, underscores, punctuation or funny business!
Create a directory called /srv/nextcloud/$USER.
Fill out a nextcloud stanza on the nextcloud KVM guest yml file you want to use. Obviously, for the user field choose the user you created in the earlier step.

In this step, you’ll need to choose a domain name (be sure to create a DNS record pointing to the web proxy you will use before continuing).

And lastly, you’ll also need to add nginx_proxy and keystore stanzas.
Apply the ansible changes. Be sure to apply to key001, webproxy001 as well as nextcloud002.
On the server, run mf-nextcloud-install $USER to complete the initial installation. This install script will spit out your admin password.

Post install clean ups

Unfortunately, not everything is fully automated. After the install:

Edit the file /srv/nextcloud/$USER/config/config.php and change the trusted_domains parameter from localhost to the site domain.
Log in as admin, click Settings -> Basic Settings and change the background job to “cron” (the systemd service and timer should be set up automatically).

Enable and start the cron service:

systemctl enable $USER-nextcloud-cron.timer
systemctl start $USER-nextcloud-cron.timer

Settings

Right now, downloads are limited to 4GB per file.

This setting happens in several locations:

On the proxy server (nginx), we have proxy_buffering set to off to avoid a limit imposed by the proxy.
On the proxy server (nginx), we have client_max_body_size set to 4g
On the nextcloud server (nginx), we have client_max_body_size set to 4g
On the nextcloud server (nextcloud), we have max_chunk_ize set to 4g. See future changes in the max_chunk_size setting.

Maintenance

Using the Nextcloud command line tool `occ`

For convenience, there is an mf-nextcloud-occ-wrapper that provides easy access to the occ script within a nextcloud instance. The occ script is the Nextcloud command line client. You can run useful commands like:

mf-nextcloud-occ-wrapper $USER user:resetpassword admin
mf-nextcloud-occ-wrapper $USER maintenance:mode --on
mf-nextcloud-occ-wrapper $USER maintenance:mode --off
mf-nextcloud-occ-wrapper $USER list

Resetting multi-factor authentication (2FA/MFA)

First, check the status of the user:

mf-nextcloud-occ-wrapper mayfirst twofactorauth:state <uid>

Then, disable each provider:

mf-nextcloud-occ-wrapper mayfirst twofactorauth:disable <uid> <provider>

For example:

mf-nextcloud-occ-wrapper mayfirst twofactorauth:disable joeuser backup_codes
mf-nextcloud-occ-wrapper mayfirst twofactorauth:disable joeuser totp

Getting into the database via the cli

Each instance has a ~/.pgpass file containing the connection credentials to access the database. Unfortunately, the psql client defaults to the localhost and our postgres databases are on a remote host so, even though a pgpass file exists, you still have to specify the host which is tedious (you can always check the contents of the ~/.pgpass file to see the host address).

For the mayfirst instance this step is facilitated by a psql convenience script. You can quickly log in to the Nextcloud database for the mayfirst instance by following these steps:

ssh root@nextcloud002.mayfirst.org
su - mayfirst
./psql

Upgrade

To see a list of available releases, visit the Nextcloud Github releases page.

Note: You can only upgrade Nextcloud one major version at a time. So if you are upgrading from version 3 to version 6, you have to repeat this process to go from 3 -> 4, 4 -> 5, and finally 5 -> 6.

To upgrade an instance:

Alter the ansible stanza for the site you want to upgrade in the nextcloud ansible yml file, change the nextcloud version number to the target version
Ensure May First Auth App has it’s info.xml file updated to be compatible with the version Nextcloud. If not, update it and push the changes to code.mayfirst.org. We will pull in the most recent version.
Manually download the new Nextcloud version on the server to /usr/local/src/nextcloud (you may need to delete old versions). On the Nextcloud server: NEXTCLOUD_VERSION=PUT_VERSION_HERE nextcloud-download
Handle any app errors from the step above - if an app is not compatible you will get an error.
Patch the code:
- To avoid “Account name or email” login message sed -i 's/Account name or email/Account name/' dist/core-login.js
- To avoid losing Favorites, comment out this code
- To avoid the Nextcloud warning on community installs. See issue #29.

--- lib/private/Notification/Manager.php.orig        2024-10-03 14:57:13.619887534 +0000
+++ lib/private/Notification/Manager.php     2024-06-25 09:58:12.000000000 +0000
@@ -285,7 +285,6 @@
         * {@inheritDoc}
         */
        public function isFairUseOfFreePushService(): bool {
+               return TRUE;
                $pushAllowed = $this->cache->get('push_fair_use');
                if ($pushAllowed === null) {

Make a backup of /srv/nextcloud/$USER/config/config.php. Nextcloud will rewrite this file and sometimes that goes hay wire.
Start the offline clock - the next step will take Nextcloud down.
Manually edit the /srv/nextcloud/$USER/config/config.php file to change maintenance mode to TRUE.
Run mf-nextcloud-backup-database $USER
Apply ansible changes to the server: sower --live playbook --tags nextcloud-upgrade nextcloudNNN.mayfirst.org
Update the nginx symlink: mf-nextcloud-set-symlink $USER
Reload php: systemctl reload php8.4-fpm
Run mf-nextcloud-occ-wrapper $USER upgrade
Set maintenance mode to FALSE in the config file.
Log in as admin and check warning when you click the “Administration Settings” link. You may want to run:
mf-nextcloud-occ-wrapper $USER db:add-missing-indices
mf-nextcloud-occ-wrapper $USER db:add-missing-columns
Uncomment the cleanTags function that was commented out in the earlier step for avoiding lost favorites
Restart notify push: systemctl restart $USER-notifypush

Restore

If a backup goes bad, here are steps to re-do it:

Release service advisory
Put site in maintenance mode
Make fresh backup (mf-nextcloud-backup-database $USER)
Change version number back to old version number
Apply ansible changes to the server.
Drop existing database and re-create:
- ssh postgres@psql003.mayfirst.cx
- dropdb mayfirst (if you get a message about open sessions, try: SELECT pg_terminate_backend(pid) FROM pg_stat_activity WHERE pid <> pg_backend_pid() AND datname = 'mayfirst';)
- createdb -O mayfirst mayfirst
Confirm mayfirst user can still login and access an empty database
- su - mayfirst
- bash
- ./psql
Restore database
- su - mayfirst
- bash
- cat /srv/nextcloud/$USER/backups/$BACKUPFILE | ./psql
Repeat upgrade

Move database to new database host

Assuming old database is psql001.mayfirst.cx and new database is psql003.mayfirst.cx:

Create user and database on new database host (from Nextcloud server as root):

ssh postgres@psql003.mayfirst.cx "psql -c \"CREATE USER mayfirst WITH PASSWORD 'GET-OLD-PASSWORD-FROM ~/.pgpass file'\""
ssh postgres@psql003.mayfirst.cx "createdb -O mayfirst mayfirst"

Edit ~/.pgpass and add line for new server (with same password)
Edit ~/psql and update database name.
Put nextcloud in maintenance mode
Make backup: mf-nextcloud-backup mayfirst
Chown backup to non-priv user: chown mayfirst /path/to/backup
(As non privileges user): Import backup: cat /path/to/backup | ./psql
Edit config.php, removing maintenance mode and also changing database host.

Nextcloud

System design

Setting up an instance

Post install clean ups

Settings

Maintenance

Using the Nextcloud command line tool `occ`

Resetting multi-factor authentication (2FA/MFA)

Getting into the database via the cli

Upgrade

Restore

Move database to new database host

Table of Contents

Pages in Categories

Tags

Nextcloud

System design

Setting up an instance

Post install clean ups

Settings

Maintenance

Using the Nextcloud command line tool occ

Resetting multi-factor authentication (2FA/MFA)

Getting into the database via the cli

Upgrade

Restore

Move database to new database host

Table of Contents

Pages in Categories

Tags

Using the Nextcloud command line tool `occ`