Avoid traversing excluded ancestors in shallow memory finds

[Mohamedajab]

With MemoryFileSystem.find(maxdepth=1, withdirs=True), deeply nested paths still make add_ancestor_dirs visit and slice every excluded ancestor. Start from the deepest ancestor allowed by maxdepth, then use the existing collection logic. This also handles directories implied by files and explicitly created empty directories.

Refs #1906. The change improves shallow find calls; it does not replace the flat memory store or change walk/ls.

Validation on Windows, CPython 3.14:

• Memory, filesystem-specification and mapping tests: 493 passed, 123 skipped, 1 expected failure. The new cases compare against the generic filesystem implementation across roots, depths and detail modes.
• Pre-commit (including Ruff and codespell) and git diff --check passed.
• One HTTP test teardown emitted a Windows socket warning. The complete backend/downstream matrix was not run.

Synthetic benchmark: 1,000 files sharing a deep parent, searching /data with maxdepth=1, withdirs=True; median of five calls, with equal results checked. At depths 20/100/300, upstream took 0.0110/0.0971/0.5044 seconds and this patch took 0.0010/0.0016/0.0026 seconds. This measures that shallow, deep-path case rather than general filesystem speed.

To reproduce timings on either revision:

from statistics import median
from timeit import repeat
from fsspec.implementations.memory import MemoryFileSystem

for depth in (20, 100, 300):
    fs = MemoryFileSystem(global_store=False, skip_instance_cache=True)
    parent = '/data/' + '/'.join(f'level{i}' for i in range(depth))
    for i in range(1000):
        fs.pipe_file(f'{parent}/file{i}', b'x')
    print(depth, median(repeat(
        lambda: fs.find('/data', maxdepth=1, withdirs=True),
        number=1, repeat=5,
    )))