Skip to content

Migrate Mocha and test configuration to v12 - #26194

Open
Raymond Zhao (rzhao271) with Copilot wants to merge 2 commits into
mainfrom
copilot/migrate-mocha-config-to-v12
Open

Raymond Zhao (rzhao271) with Copilot wants to merge 2 commits into
mainfrom
copilot/migrate-mocha-config-to-v12

Conversation

Copilot AI commented Oct 2, 2026 •

Copy link
Copy Markdown

Upgrade Mocha to 12.0.2 using npm 11.19 with --min-release-age 7.

  • Configuration: Update debugger entrypoints and setup-module paths; load ambient declarations for TypeScript test runs.
  • Reporting: Replace incompatible multi/JUnit reporter dependencies with Mocha’s built-in Spec/XUnit reporters, preserving PID-prefixed console output and MOCHA_FILE.
  • Completion: Flush XML before notifying the extension-test socket server, preventing truncated reports.
  • Regression coverage: Cover passing, failing, and pending XML results, output-path precedence, and extension-test completion.

Copilot AI and others added 2 commits October 2, 2026 22:19
Co-authored-by: rzhao271 <7199958+rzhao271@users.noreply.github.com>
Co-authored-by: rzhao271 <7199958+rzhao271@users.noreply.github.com>
@rzhao271
Raymond Zhao (rzhao271) marked this pull request as ready for review October 2, 2026 22:54
@rzhao271
Raymond Zhao (rzhao271) enabled auto-merge (squash) October 2, 2026 22:54
@heejaechang

Heejae Chang (heejaechang) commented Oct 7, 2026 •

Copy link
Copy Markdown

🔒 Automated review in progress — Heejae Chang (@heejaechang) is auto-reviewing this PR.

@heejaechang

Copy link
Copy Markdown

Result: ⚠️ needs-more-tests

Verification details

Verification: Isolated verification observed failures that were not classified as caused by this PR: Dependency and test discovery, Default XML and real socket completion probe. The relevant tests could not be fully run in the isolated environment; this review is not fully verified.

Summary: Offline dependency installation and TypeScript compilation passed. Targeted compiled-unit, TypeScript-unit, and functional runs passed with 4, 28, and 25 tests respectively; nested failures were intentional reporter fixtures. XML output precedence, default output, and PID-prefixed console reporting were verified. The socket-completion probe failed because my harness used Mocha’s module namespace as a constructor, not because of changed PR code. Actual socket notification after XML completion remains uncovered by the added regression tests, and Electron integration was not run.

Test runs: 4 passed, 2 failed, 1 not run

  • ⚠️ Not run | Electron single-workspace integration | npm run testSingleWorkspace
  • ❌ Failed | unrelated to this PR | Dependency and test discovery | printf 'Sandbox profile: %s\n' "$AUTOMATION_SANDBOX_PROFILE"; node --version; npm --version; git status --short; node -e "const fs=require('fs'); const p=require('./package.json'); console.log('Mocha dependency:',p.devDependencies.mocha); console.log('Test scripts:',JSON.stringify(Object.fromEntries(Object.entries(p.scripts).filter(([k])=>/test|compile/.test(k))),null,2)); for(const f of ['node_modules/mocha/package.json','node_modules/typescript/package.json','node_modules/ts-node/package.json','out/test/unittests.js','out/test/common/ciReporter.unit.test.js']) console.log(f,fs.existsSync(f)?'present':'missing');"; git ls-files 'src/test/report' 'src/test/Reporter' 'src/test/common/Reporter' 'build/.mocha*' 'types/*'
  • ❌ Failed | unrelated to this PR | Default XML and real socket completion probe | node <<'NODE'
    'use strict';
    const assert = require('assert');
    const fs = require('fs');
    const os = require('os');
    const path = require('path');
    const net = require('net');
    const Mocha = require('mocha');
    const { parseStringPromise } = require('xml2js');
    const CIReporter = require('./build/ci/scripts/ci_reporter');
    const ExitReporter = path.resolve('out/test/common/exitCIAfterTestReporter.js');
    (async () => {
    const defaultXml = await parseStringPromise(fs.readFileSync('test-results.xml', 'utf8'));
    assert.strictEqual(defaultXml.testsuite.$.tests, '28');
    assert.strictEqual(defaultXml.testsuite.$.errors, '0');
    console.log('Validated default test-results.xml from the targeted TypeScript run');
    const directory = fs.mkdtempSync(path.join(os.tmpdir(), 'python-reporter-verification-'));
    const portFile = path.resolve('port.txt');
    assert.ok(!fs.existsSync(portFile), 'fresh verification workspace should not have port.txt');
    const previousMochaFile = process.env.MOCHA_FILE;
    const server = net.createServer();
    let timer;
    try {
    await new Promise((resolve, reject) => { server.once('error', reject); server.listen({ host: '127.0.0.1', port: 0 }, resolve); });
    fs.writeFileSync(portFile, String(server.address().port));
    for (const expectedFailures of [0, 1]) {
    const output = path.join(directory, socket-${expectedFailures}.xml);
    process.env.MOCHA_FILE = output;
    let connect;
    const connected = new Promise(resolve => { connect = resolve; });
    const received = new Promise((resolve, reject) => {
    timer = setTimeout(() => reject(new Error('completion notification timed out')), 5000);
    server.once('connection', socket => {
    socket.on('error', reject);
    socket.once('data', data => {
    try { resolve({ code: data.toString(), xml: fs.readFileSync(output, 'utf8') }); } catch (error) { reject(error); }
    });
    connect();
    });
    });
    const mocha = new Mocha({ reporter: CIReporter, reporterOptions: { consoleReporter: ExitReporter } });
    const suite = Mocha.Suite.create(mocha.suite, 'Socket completion verification');
    suite.addTest(new Mocha.Test('waits for normal reporter connection', async () => { await connected; }));
    suite.addTest(new Mocha.Test('scenario result', () => { if (expectedFailures) throw new Error('expected verification failure'); }));
    suite.addTest(new Mocha.Test('pending'));
    const [failures, notification] = await Promise.all([new Promise(resolve => mocha.run(resolve)), received]);
    clearTimeout(timer);
    assert.strictEqual(failures, expectedFailures);
    assert.strictEqual(notification.code, String(expectedFailures));
    const { testsuite } = await parseStringPromise(notification.xml);
    assert.strictEqual(testsuite.$.tests, '3');
    assert.strictEqual(testsuite.$.errors, String(expectedFailures));
    assert.strictEqual(testsuite.$.skipped, '1');
    assert.ok(notification.xml.trimEnd().endsWith('</testsuite>'));
    console.log(PASS: socket code ${expectedFailures} received only with complete, parseable XML);
    }
    } finally {
    clearTimeout(timer);
    await new Promise(resolve => server.close(resolve));
    fs.rmSync(portFile, { force: true });
    fs.rmSync(directory, { recursive: true, force: true });
    if (previousMochaFile === undefined) delete process.env.MOCHA_FILE;
    else process.env.MOCHA_FILE = previousMochaFile;
    }
    })().catch(error => { console.error(error); process.exitCode = 1; });
    NODE
  • ✅ Passed | Compiled CI reporter and string utility tests | node ./node_modules/typescript/lib/tsc.js -p ./ && node -e "const fs=require('fs');const c=JSON.parse(fs.readFileSync('build/.mocha.unittests.json','utf8'));c.spec=['./out/test/common/ciReporter.unit.test.js','./out/test/common/stringUtils.unit.test.js'];fs.writeFileSync('.verification-mocha-js.json',JSON.stringify(c));" && node ./node_modules/mocha/bin/mocha.js --config ./.verification-mocha-js.json
  • ✅ Passed | TypeScript CI reporter and terminal environment collection tests | node -e "const fs=require('fs');const c=JSON.parse(fs.readFileSync('build/.mocha.unittests.ts.json','utf8'));c.spec=['./src/test/common/ciReporter.unit.test.ts','./src/test/interpreters/activation/terminalEnvVarCollectionService.unit.test.ts'];fs.writeFileSync('.verification-mocha-ts.json',JSON.stringify(c));" && node ./node_modules/mocha/bin/mocha.js --config ./.verification-mocha-ts.json
  • ✅ Passed | Functional path tests through performance reporter configuration | node -e "const fs=require('fs');const c=JSON.parse(fs.readFileSync('build/.mocha.performance.json','utf8'));c.spec=['./out/test/common/platform/fs-paths.functional.test.js'];fs.writeFileSync('.verification-mocha-performance.json',JSON.stringify(c));" && MOCHA_FILE=.verification-ignored.xml VSC_PYTHON_CI_TEST_PARALLEL=1 node ./node_modules/mocha/bin/mocha.js --config ./.verification-mocha-performance.json > .verification-performance.log 2>&1 && node -e "const fs=require('fs');const assert=require('assert');const {parseStringPromise}=require('xml2js');const text=fs.readFileSync('.verification-performance.log','utf8');console.log(text.replace(/\x1b[[0-9;]*m/g,'').split('\n').slice(-35).join('\n'));assert.match(text,/\d+ /);assert.ok(!fs.existsSync('.verification-ignored.xml'));parseStringPromise(fs.readFileSync('xunit-test-results.xml','utf8')).then(({testsuite})=>{assert.ok(Number(testsuite.$.tests)>0);assert.strictEqual(testsuite.$.failures,'0');assert.strictEqual(testsuite.$.errors,'0');console.log('Validated PID-prefixed Spec output and performance XML output precedence:',testsuite.$);}).catch(e=>{console.error(e);process.exitCode=1;});"
  • ✅ Passed | Offline dependency bootstrap | npm ci --offline
⚠️ Electron single-workspace integration diagnostic output
Electron tests requiring a downloaded VS Code instance are unsupported in this verification environment.
❌ Dependency and test discovery diagnostic output
Sandbox profile: typescript
v22.21.1
10.9.4
Mocha dependency: ^12.0.2
node_modules/mocha/package.json missing
node_modules/typescript/package.json missing
node_modules/ts-node/package.json missing
out/test/unittests.js missing
out/test/common/ciReporter.unit.test.js missing
fatal: not a git repository (or any parent up to mount point /)
[container exit=128]
❌ Default XML and real socket completion probe diagnostic output
Validated default test-results.xml from the targeted TypeScript run
TypeError: Mocha is not a constructor
    at [stdin]:40:27
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
[container exit=1]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved via Review Center.

@heejaechang Heejae Chang (heejaechang) added the review-auto:approved Automated review: no blocking findings (approval posted). label Oct 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review-auto:approved Automated review: no blocking findings (approval posted).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants