Concurrent automatic Unix-socket server startup can fail with 'File exists (os error 17)' #2906

With a private Unix socket, four clients starting together can fail with
Server startup failed: File exists (os error 17). Observed with the official v0.18.0
release binary (aarch64-apple-darwin, macOS 26): 6 of 80 clients failed in 20 rounds of
4 concurrent starts. We first hit it on 0.15.0 (4 of 80); the relevant code in
src/server.rs is unchanged in v0.18.0.

Reproduce with the following standalone script, supplying a release
binary. Every trial owns its HOME, artifact cache, socket, TMPDIR and process
groups; no existing server is contacted or stopped. Logs remain under the printed
private directories. SCCACHE_NO_DAEMON=1 keeps startup children in their owned
groups for cleanup. --zero-stats drives automatic startup without needing a
compiler. The barrier synchronizes clients without sleeps.

SCCACHE_REPRO_BINARY=/path/to/sccache-0.18.0 python3 - <<'PYREPRO'
import concurrent.futures, json, os, pathlib, signal, subprocess, tempfile, threading
binary = os.environ['SCCACHE_REPRO_BINARY']
failures = 0
for trial in range(20):
    root = pathlib.Path(tempfile.mkdtemp(prefix='sccache-race-', dir='/tmp'))
    (root / 'tmp').mkdir()
    (root / 'cache').mkdir()
    env = {k: v for k, v in os.environ.items()
           if not k.startswith('SCCACHE_')}
    env.update(HOME=str(root), XDG_CONFIG_HOME=str(root / 'config'),
               SCCACHE_DIR=str(root / 'cache'), TMPDIR=str(root / 'tmp'),
               SCCACHE_SERVER_UDS=str(root / 'server.sock'),
               SCCACHE_NO_DAEMON='1', SCCACHE_IDLE_TIMEOUT='0', SCCACHE_LOG='debug')
    children = []
    barrier = threading.Barrier(4)
    def launch(i):
        with (root / f'client-{i}.log').open('w') as log:
            barrier.wait(timeout=10)
            child = subprocess.Popen([binary, '--zero-stats'], env=env,
                stdout=log, stderr=log, start_new_session=True)
            children.append(child)
            return child.wait(timeout=60)
    try:
        with concurrent.futures.ThreadPoolExecutor(4) as pool:
            codes = list(pool.map(launch, range(4)))
        failures += sum(code != 0 for code in codes)
        print(json.dumps(dict(root=str(root), codes=codes)), flush=True)
    finally:
        for child in children:
            try: os.killpg(child.pid, signal.SIGKILL)
            except ProcessLookupError: pass
        for child in children:
            child.wait(timeout=10)
    for log in sorted(root.glob('client-*.log')):
        for line in log.read_text().splitlines():
            if 'File exists' in line:
                print(line)
print('Failed clients:', failures)
PYREPRO

Root cause: src/server.rs at v0.18.0
unconditionally removes the Unix socket path before binding. Two startup
children can unlink each other's socket and race on bind. The child is launched
via current_exe, so wrapping the client cannot coordinate that critical section
without also serializing compiler requests.

Proposed fix: take an exclusive advisory flock on an owner-only sibling
<socket>.lock around live-listener probing, stale-socket reclamation and bind.
Keep its inode across starts and crashes. If a listener is live, notify startup
success and exit the losing starter. Remove only an owned socket after a failed
connection establishes absence/refusal. Preserve paths on uncertain probes or
unsafe metadata. Release the lock before startup notification and serving
compiler requests. TCP and abstract-socket behavior stays unchanged. Suggested regression
coverage: check 20×4 starts, exactly one server per round, live socket inode
preservation, unsafe paths, and reclaiming a socket left by a killed server.

Observed failing output (0.18.0):

sccache: error: Server startup failed: File exists (os error 17)
[ERROR sccache::server] failed to start server: File exists (os error 17)