SMB external storage: files get removed from Nextcloud when the server is up but the share is missing #65154

⚠️ This issue respects the following points: ⚠️

• This is not a troubleshooting question, general support matter, or webserver/proxy problem, but likely a bug (if unsure, ask the Community Help Forum).
• This issue is not already reported on Github OR solved at the Community Help Forum (I've searched!).
• I'm using a maintained major version of Nextcloud Server and tested against the latest patch level. (Supported major versions and current patch levels).
• I agree to follow Nextcloud's Code of Conduct.
• I've tried my best to provide clear reproduction steps that someone unfamiliar with this bug could use to reproduce it.

Bug description

I have an SMB external storage pointing at a share on a TrueNAS box, on an encrypted dataset. When the NAS reboots, the dataset stays locked until I unlock it, so the SMB server is up and answering but the share doesn't exist. The NAS emails me about unexpected reboots, so I usually unlock it before I next use Nextcloud.

But in the time before I unlocked it, Nextcloud still lost my photo folder (~34k files) every time. The Memories app's background indexing touches that folder every few minutes, and each time Nextcloud dropped the whole folder from its file index, together with the Memories data and the pre-generated previews. After unlocking, the next scan added everything back under new file IDs, so it looked fine to me, apart from thumbnails loading slowly because the ones pre-generated by the Preview Generator app were gone. Looking at old backups, this had happened at least four times since spring (on 32, 33, 34 and 35) without me noticing.

This time I used Nextcloud before unlocking the dataset, and then the rest of the mount went too: over a few hours it went from about 1.5 million entries in the index to a couple dozen. That's when I noticed. I had to restore the database from a backup and regenerate all thumbnails from scratch.

What makes it surprising is that Nextcloud handles the NAS being completely off just fine. When the host is down or refuses the connection, the storage is marked as unavailable and nothing gets removed. Only the "server is up but the share is gone" case is treated as if every file had been deleted.

There's precedent for this with Local external storage. When its path was unmounted, Nextcloud reported it as "not found" instead of "storage not available", and desktop clients deleted their local copies as a result (#39706). That was fixed in #39707, but only for Local storage. Since the SMB case also reports the files as not found, I'd expect sync clients could react the same way here, but I haven't verified that. I also haven't looked at how the other external storage backends handle their share or root going missing.

Steps to reproduce

1. On an SMB server, create a share (here: test) with a few folders and files, e.g. sub/a.txt.
2. In Nextcloud, with the php-smbclient extension installed, add it as external storage:
   • UI: Administration settings → External storage → SMB/CIFS, folder name NAS, share test, available for alice
   • CLI: occ files_external:create /NAS smb password::password -c host=<server> -c share=test -c user=<user> -c password=<password>, then occ files_external:applicable --add-user alice <id>
   • As alice, open NAS/sub: a.txt is there
3. Make the share disappear while the SMB server stays up, e.g. remove the [test] section from smb.conf , or on TrueNAS lock the dataset or disable the share.
   • smbclient //<server>/test now fails with NT_STATUS_BAD_NETWORK_NAME
4. Let Nextcloud touch the folder:
   • UI: open NAS/sub as alice
   • CLI: occ files:scan --path=/alice/files/NAS/sub
   • a.txt is removed from Nextcloud's file index, the same as if it had been deleted. With a bigger tree, whole folders disappear this way while the share is missing.
5. Bring the share back and rescan.
   • a.txt shows up again, but with a new file ID.

Expected behavior

A missing share should be treated the same as an unreachable server: mark the storage as unavailable, and don't touch the file index until it's back.

Nextcloud Server version

35

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.5

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 31 to 32)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

• Default user-backend (database)
• LDAP/ Active Directory
• SSO - SAML
• Other

Configuration report

{
    "system": {
        "htaccess.RewriteBase": "/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "/var/www/html/apps",
                "url": "/apps",
                "writable": false
            },
            {
                "path": "/var/www/html/custom_apps",
                "url": "/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": 25,
        "mail_smtpauthtype": "LOGIN",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "upgrade.disable-web": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED***",
            "nextcloud"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "35.0.1.1",
        "overwrite.cli.url": "***REMOVED***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "overwriteprotocol": "https",
        "defaultapp": "",
        "mail_sendmailmode": "smtp",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "",
        "mail_smtpauth": false,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "enforce_theme": "",
        "data-fingerprint": "***REMOVED***",
        "davstorage.request_timeout": 120,
        "memories.exiftool": "/var/www/html/custom_apps/memories/bin-ext/exiftool-amd64-glibc",
        "memories.vod.path": "/var/www/html/custom_apps/memories/bin-ext/go-vod-amd64",
        "memories.vod.ffmpeg": "/usr/bin/ffmpeg",
        "memories.vod.ffprobe": "/usr/bin/ffprobe",
        "memories.index.mode": "3",
        "preview_max_x": 1024,
        "preview_max_y": 1024,
        "enabledPreviewProviders": [
            "OC\\Preview\\MP3",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\Krita",
            "OC\\Preview\\Imaginary",
            "OC\\Preview\\Movie"
        ],
        "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
        "memories.index.path": "/<smb mount>/<photos folder>/",
        "allow_local_remote_servers": true,
        "maintenance": false,
        "loglevel": 2,
        "memories.db.triggers.fcu": true,
        "app_install_overwrite": [
            "limit_login_to_ip"
        ]
    }
}

List of activated Apps

Enabled:
  - activity: 8.0.0
  - app_api: 35.0.0
  - appstore: 2.0.0-dev.0
  - bruteforcesettings: 8.0.0
  - cloud_federation_api: 2.0.0-dev.0
  - cookbook: 0.11.10
  - cospend: 4.1.3
  - dav: 2.0.0-dev.1
  - external: 10.0.0
  - federatedfilesharing: 2.0.0-dev.1
  - federation: 2.0.0-dev.0
  - files: 3.0.0-dev.0
  - files_downloadlimit: 5.3.0
  - files_external: 2.0.0-dev.0
  - files_lock: 35.0.0
  - files_pdfviewer: 8.0.0
  - files_reminders: 2.0.0-dev.0
  - files_sharing: 2.0.0-dev.1
  - files_trashbin: 2.0.0-dev.0
  - files_versions: 2.0.0-dev.0
  - gpxpod: 8.3.1
  - limit_login_to_ip: 4.4.1
  - logreader: 8.0.0
  - lookup_server_connector: 2.0.0-dev.0
  - memories: 9.0.1
  - nextcloud_announcements: 7.0.0
  - notes: 6.1.0
  - notifications: 8.0.0
  - oauth2: 2.0.0-dev.0
  - office: 1.1.0
  - password_policy: 7.0.0
  - previewgenerator: 5.15.0
  - profile: 2.0.0-dev.0
  - provisioning_api: 2.0.0-dev.0
  - richdocuments: 12.0.1
  - serverinfo: 7.0.0
  - settings: 2.0.0-dev.0
  - sharebymail: 2.0.0-dev.0
  - sharing: 1.0.4
  - survey_client: 7.0.0
  - text: 9.0.0
  - theming: 3.0.0-dev.0
  - twofactor_backupcodes: 2.0.0-dev.0
  - twofactor_totp: 17.1.0
  - twofactor_webauthn: 2.7.1
  - unroundedcorners: 1.1.6
  - updatenotification: 2.0.0-dev.0
  - viewer: 8.0.0
  - webhook_listeners: 2.0.0-dev.0
  - workflowengine: 3.0.0-dev.0

Nextcloud Signing status

Core: no errors.

files_external: 1 errors found:
  - INVALID_HASH:
    - lib/Lib/Storage/SMB.php:
      - expected: 6bfee85ff4b42b0fa90d91f3074b53ca44682b50e1931da24032b26b7db2ded6e04cfb6ef6b6c38e81b5af5e8219c752dc88c8f516ec00090b691f375bbec72f
      - current: a5967b9108283c53dbeecfdee742be0bb5a10a09a528cd6b3900c6039ba01b4712b0a12c6c48a89e2a94e8ae90fab6a58d9d10e79bb951fbc9f473c039ee8ff9

(This is the patch from "Additional info", applied locally. The problem happened on unmodified code, before I applied it.)

Nextcloud Logs

Nothing relevant

Additional info

I debugged this with the help of Claude, which did most of the code analysis. I checked the fix myself on my instance.

From what we found, with the php-smbclient backend a missing share (NT_STATUS_BAD_NETWORK_NAME) comes back as a plain "not found" for every path. An unreachable server, on the other hand, is reported as "storage not available". So Nextcloud concludes that every file it looks at on the share has been deleted, and drops it from the index. In our case Memories walks the photo folder every few minutes, which is why that folder went every time, and browsing the mount this time took the rest with it. #50298 made this easier to hit, since a failed stat now counts as "changed".

The smbclient-binary backend (without php-smbclient) reports a missing share as a connection problem instead, so it isn't affected.

The fix I'm running: when something isn't found, check whether the share root is still reachable, and if it isn't, mark the storage as unavailable. I locked the dataset again with this applied. The storage was marked unavailable, nothing was removed, and it recovered on its own after unlocking.

--- a/apps/files_external/lib/Lib/Storage/SMB.php
+++ b/apps/files_external/lib/Lib/Storage/SMB.php
@@ -176,6 +176,7 @@
 		} catch (InvalidArgumentException $e) {
 			$this->throwUnavailable($e);
 		} catch (NotFoundException $e) {
+			$this->assertRootAvailable($path, $e);
 			throw new \OCP\Files\NotFoundException($e->getMessage(), 0, $e);
 		} catch (ForbiddenException $e) {
 			// with php-smbclient, this exception is thrown when the provided password is invalid.
@@ -194,6 +195,28 @@
 	}
 
 	/**
+	 * A missing share (NT_STATUS_BAD_NETWORK_NAME, e.g. when the backing dataset is locked)
+	 * is reported by php-smbclient as a plain NotFound for every path. Before reporting a path
+	 * as not found, make sure the storage root itself is reachable, otherwise the scanner
+	 * would remove the whole storage from the cache.
+	 *
+	 * @throws StorageNotAvailableException
+	 */
+	private function assertRootAvailable(string $fullPath, \Exception $e): void {
+		$rootPath = $this->buildPath('');
+		if (ltrim($fullPath, '/') !== ltrim($rootPath, '/')) {
+			try {
+				$this->share->stat($rootPath);
+				return;
+			} catch (\Exception $rootException) {
+				$e = $rootException;
+			}
+		}
+		$this->logger->warning('Storage root not reachable, marking storage as unavailable', ['exception' => $e]);
+		throw new StorageNotAvailableException($e->getMessage(), StorageNotAvailableException::STATUS_ERROR, $e);
+	}
+
+	/**
 	 * @throws StorageAuthException
 	 */
 	protected function throwUnavailable(\Exception $e): never {
@@ -272,6 +295,7 @@
 			$this->logger->error('Error while getting folder content', ['exception' => $e]);
 			throw new StorageNotAvailableException($e->getMessage(), (int)$e->getCode(), $e);
 		} catch (NotFoundException $e) {
+			$this->assertRootAvailable($path, $e);
 			throw new \OCP\Files\NotFoundException($e->getMessage(), 0, $e);
 		}
 	}

I can open a PR with this if it looks like the right approach.