Repository navigation
In-memory virtual file overlay over LSP without faking didOpen #64611
Description
Activity
Hrm.... feels a lot like you'd be well-served by content-mapping
.tsfiles just to map all the inner template strings - but if we allowed that, then we'd have to worry about coordinating chaining them...Content mappers don't really fit our architecture for a few reasons:
First, Angular components are defined in
.tsfiles—often with inline templates where there is no separate.htmlfile at all—and content mappers can't be registered for.tsfiles.Second, content mappers are designed around transforming a single file in isolation during parsing (
{ fileName, content }). Generating a component's Type Check Block (TCB) requires cross-file program knowledge (which directives, components, and pipes are in scope via imports/NgModules, their input/output types, etc.). You can't generate the TCB by looking at a template or component file on its own.Even setting those limitations aside, we'd basically use none of the features content mappers provide because the mapping model doesn't fit how TCBs work.
.ngtypecheck.tsfiles aren't a clean "this section of the file is TS code" mapping—templates have their own expression syntax that gets compiled into synthetic TS statements with nested spans and custom diagnostic post-processing. Our language server already runs its own compiler instance to generate the.ngtypecheck.tsfiles in memory and handle all the position/diagnostic translation, so we'd be using the content mapper API for almost nothing. We just need a way to push those already-generated in-memory.ngtypecheck.tsfiles into TS-Go's VFS without fakingdidOpenand pollutingopenClientFiles.Our language server already runs its own compiler instance to generate the .ngtypecheck.ts files in memory and handle all the position/diagnostic translation, so we'd be using the content mapper API for almost nothing
Right, exactly, that's the point, if you could return all that directly (eg, via content mapper API), then in theory you don't need to handle any of the other translation stuff at the higher LSP level, since the TS LSP knows how to apply mappings from the content mapper API to LSP operations already. Content mappers are also, canonically, a long-running process - they could easily be your template server, waiting to just serve up the compilation result from the whole-program inspection you already did on last underlying FS change/editor edit. Basically allowing you to invert control - the normal TS LSP could load the angular content mapper and fetch diagnostics/definitions/refs/etc, instead of angular wrapping the TS LSP and managing state/forwarding calls-with-changes.
and content mappers can't be registered for .ts files.
Not just yet due to some hesitation on our part...but apparently they can be for
.ng.tsfiles until this PR merges if you wanted to quickly try it and see if the model fits...Just a thought - since, the direction you're going as-is it's starting to feel like you just need a lot of the API for local state management exposed over LSP (which, like, why not just use the API directly then).
Even with a long-running content mapper process, we couldn't invert control and rely on the TS LSP's span mapping to serve template features directly. Angular template operations aren't a 1:1 pass-through to a single offset in
.ngtypecheck.ts—many require multi-hop LSP queries (like resolvingtypeDefinitionon a synthetic TCB variable and then queryinghoveron the resulting class declaration), fanning out across multiple matched directives on a single attribute, rewriting display text, or computing completions and renames from Angular's template AST and component scope metadata rather than TypeScript. There's also no 1:1 file mapping with.html, since.ngtypecheck.tsis paired with the.tsfile and can aggregate multiple external.htmlfiles and inline templates.As for using the
tsgo --apidirectly instead of LSP,updateSnapshothas the virtual file overlay support we want, but--apidoesn't expose the language service requests (hover,definition,rename,references, etc.) that we need to query on the generated.ngtypecheck.tsfiles.That said, this isn't a hard blocker for us. Sending
didOpen/didChangefor the.ngtypecheck.tsfiles works today. This is a nice-to-have to avoid polluting the open-file state and keeping projects pinned open longer than necessary, especially if it's not handled perfectly.As for using the tsgo --api directly instead of LSP, updateSnapshot has the virtual file overlay support we want, but --api doesn't expose the language service requests (hover, definition, rename, references, etc.) that we need to query on the generated .ngtypecheck.ts files.
Oh yeah, that's an API hole I think we wanna fill - it was available before and should be now, we just haven't gotten to it yet (I think we only exposed completions over the API like a few weeks ago).
LanguageServiceonly currently exposesgetImportAdderEdits[ForSymbols],getReferencedSymbolsForNode,getSignatureUsage, andgetCompletionsAtPosition- rename and quickinfo are definite notable absences. If you happen to have a full list of the missing backing APIs from TS6 that you'd need to use the API, we can prioritize getting those useful ones in first (I think Isabel Duan (@iisaduan) is actually currently working on generating a full API diff to see what we're currently missing?). I think we wanna avoid polluting the (generally very standard) LSP protocol with too many custom API-esque messages.Oh yeah, that's an API hole I think we wanna fill - it was available before and should be now, we just haven't gotten to it yet (I think we only exposed completions over the API like a few weeks ago)
Oh that'd be nice actually.
(AI data mining):
Looking at the current API surface in
packages/typescript/src/api/, there are two main gaps for us:Because we translate template positions to positions in our virtual
.ngtypecheck.tsfiles and query TypeScript'sLanguageServiceto implement our LSP handlers (often inspecting multiple TCB nodes or rewriting the result before returning it), our current prototype also relies on the following handlers in LSP not implemented inProject.languageService:getQuickInfoAtPositiongetDefinitionAtPositionandgetDefinitionAndBoundSpangetTypeDefinitionAtPositiongetReferencesAtPosition/findReferencesgetRenameInfoandfindRenameLocationsgetSignatureHelpItemsgetCompletionEntryDetails(andreplacementSpanonCompletionEntry)
Our existing TS6 language service also uses a couple of additional APIs that aren't in our prototype yet, depending on how far we go with feature parity:
getCodeFixesAtPositionandgetCombinedCodeFix(to delegate missing-member quick fixes on component classes from templates)includeCompletionsForModuleExportsongetCompletionsAtPosition(to discover out-of-scope standalone directives/pipes for auto-import completions and quick fixes)
Second, we need a way for virtual
.ngtypecheck.tsfiles to persist across LSP turns when sharing the LSP server's state. Right now,getCurrentLanguageServerSnapshot()insession.godoesn't accept afileSystemoverlay, and callingsnapshot.update({ fileSystem: { kind: "layer", files } })creates a separate snapshot that only lives on the API side. Two issues come out of that:- When a user edits a
.tsfile in the editor and we callgetCurrentLanguageServerSnapshot()to pick up the LSP server's latest changes, the returned snapshot resetsfileSystemtonil, wiping out all of our previously generated.ngtypecheck.tsfiles unless we re-send every virtual file in the project on every update. - Cross-file LSP features handled directly by the TS LSP server on
.tsfiles (like Find All References or Rename on a component property) won't see references inside.ngtypecheck.tsunless the LSP server's own state knows about those virtual files (or unless we intercept.tsreferences/renames too and run them against our layered API snapshot).
I’m working on typed-sql, and I think we have another use case for this. The idea is to write SQL inside tagged templates in regular .ts/.tsx files and infer the query’s result and parameter types. We do that by generating an in-memory TS view with the inferred type arguments, mapping positions back to the original source.
The part we’re missing is having those types available to the shared TS language server, so code consuming the query results gets the correct hover, completions and diagnostics. Running a separate provider alongside the built-in one currently gives us conflicting hovers (context).
I tested .ts/.tsx content mappers on
7.1.0-dev.20261007.1and still getTS18066, while the same setup works with a custom extension. I understand the concern around chaining multiple mappers, but this seems like a use case that would benefit from allowing it.
Would content mapping be the intended way to support this? Or is there another API planned to make the transformed source available to the shared language server while keeping positions and edits mapped to the original file?FWIW, content mapping for tagged template strings is something I plan to investigate for a future release. For now, the separate server with an API connection is the intended way, plus #64583 if you need to suppress/change a built-in hover.
I think, combined with the missing LS APIs you listed above, #64679 will give you the kind of state management you want over API connections? This way you can avoid resending unchanged FS data after each explicit LSP update.
Oh, yea that looks like it'll work nicely
🔍 Search Terms
virtual file
setVirtualFile
RequestFileSystem
openClientFiles
didOpen didClose
temporary file update
✅ Viability Checklist
⭐ Suggestion
Add an overlay mechanism over the LSP server interface (
tsgo --lsp) to push in-memory file contents directly into TS-Go's VFS (for example, a custom notification likets/setVirtualFilewith{ path, content }, wherenullclears the overlay) without treating the files as client-opened documents.📃 Motivating Example
We're using
tsgo --lsp(with the API session pipe) as our backend diagnostics and type-checking engine for the Angular Language Service. To type-check templates, Angular generates synthetic in-memory TypeScript files containing Type Check Blocks (likeapp.component.ngtypecheck.ts) that TS-Go needs to type-check alongside user code.Right now, we feed these into TS-Go by sending synthetic
textDocument/didOpenandtextDocument/didChangenotifications. While that works, treating these files as client-opened documents pollutesopenClientFiles, artificially bumps project retention ref-counts inProjectService, and forces us to manage synthetic open/close lifecycle bookkeeping for files that aren't actually open in the editor.The API client already has some support for layered VFS snapshots (
RequestFileSystemwith{ kind: "layer" }andrunWithTemporaryFileUpdate), but we don't have an equivalent way to push un-opened virtual files over the LSP connection.An overlay notification over LSP (something like
setVirtualFile(path, content | null)) would let us push our generated type-check files directly into TS-Go's VFS withoutopenClientFilespollution or open/close bookkeeping.💻 Use Cases
To feed Angular's generated template type-checking code (
.ngtypecheck.tsfiles containing Type Check Blocks) into TS-Go in memory.We run
tsgo --lspas an out-of-proc semantic and diagnostics engine for both our dev server and our separate Angular Language Server. As users edit templates, we generate and update these synthetic.ngtypecheck.tsfiles so TS-Go can type-check them alongside the user's TypeScript code and answer semantic queries (hover, definition, completions). None of these synthetic files exist on disk, and they change frequently as templates are edited.Over LSP, the only current way to provide in-memory content is via
textDocument/didOpenandtextDocument/didChange. BecausedidOpenis built for user-facing editor buffers, using it for internal synthetic files causes a few headaches:openClientFiles, adding unnecessary tracking overhead for files the user never opened.ProjectServiceretention ref-counts, keeping configured projects alive longer than they should be or creating awkward project lifecycle behavior.didOpen/didClosebookkeeping layer just to keep TS-Go's internal project state clean.The API client recently added layered snapshot file systems (
RequestFileSystemwith{ kind: "layer" }andrunWithTemporaryFileUpdate), but that's scoped to immutable snapshots in the Node API client rather than a long-runningtsgo --lspserver session answering standard LSP requests.In our language service facade, we currently fake the document lifecycle: we track synthetic version numbers, send
textDocument/didOpenwhen generating a TCB,textDocument/didChangeas the template updates, and sendtextDocument/didClosewhen components are removed or projects unload. It works functionally, but it's fragile and forces us to manage artificial lifecycle bookkeeping for purely ephemeral compiler artifacts.