Skip to content

Implement multi-language support - #130

Merged
msujew merged 4 commits into
mainfrom
msujew/multi-language-support
Oct 22, 2026
Merged

msujew merged 4 commits into
mainfrom
msujew/multi-language-support

Conversation

@msujew

@msujew msujew commented Jul 23, 2026 •

Copy link
Copy Markdown
Member

Closes #96

Does as the title says, similar to how it is outlined in the issue:

  • Adds a fastbelt/cmd package adopters can use to write their own CLIs to generate Fastbelt code.
  • Adds a LanguageSelector service that is used to determine during runtime what language a URI/document belongs to.
  • Implements a switch-case based selector in the parser and lexer to circumvent the need for a separate service instance per language.
  • Adds a multilang example that shows that this is working as expected (including parser, lexer and completion tests).

FYI, while this has a Plugins field in the API, plugins aren't actually supported in this PR. This will need a follow-up.

@cdietrich

Copy link
Copy Markdown

wondering about the common lexical scope and an option to have a lexer per entry rule
#131

@msujew
msujew force-pushed the msujew/multi-language-support branch from 78988e1 to 9f6d652 Compare July 27, 2026 14:14
github-actions[bot]

This comment was marked as outdated.

@msujew

msujew commented Jul 27, 2026 •

Copy link
Copy Markdown
Member Author

@cdietrich Thanks for the input. I plan to finish implementing this once #127 lands, since it refactors a lot of the lexer infrastructure.

@ssmifi ssmifi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks solid, thanks. I have a couple of questions to get a better understanding before merging and some documentation seems to be stale.

Comment thread cmd/fastbelt/generate.go Outdated
Comment thread cmd/fastbelt/generate.go Outdated
Comment thread cmd/build.go Outdated
Comment thread cmd/build.go Outdated
Comment thread cmd/build.go Outdated
Comment thread workspace/builder.go
Comment thread workspace/initializer.go
Comment thread document.go Outdated
Comment thread language_selector.go Outdated
Comment thread language_selector.go Outdated
Comment thread examples/arithmetics/services.go Outdated
Comment thread cmd/build.go
Comment thread internal/languages/multilang/multilang.fb Outdated
Comment thread document.go Outdated

@sailingKieler sailingKieler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @msujew for this effort! Great step forward.

Actually, I doubt the necessity and benefit for the additional language build customization for the multi language aspect. Other concerns might justify that.
I left some comments inline.

The other aspect is the documentation of impact of multiple entry rules on the lexer.
That is something I didn't expect in the first place, and others (newbies) probably also don't. See also below.

Comment thread cmd/build.go Outdated
Comment thread language_selector.go

// LanguageSelector resolves a document URI to the index of the owning language
// (into the configured languages), or -1 if none match.
type LanguageSelector interface {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On the first impression this felt a bit over engineered to me.
After thinking about it for a moment or tow, I came to the conclusion that this mainly bridges the lack of multi-bindings in our dependency container, here, multiple DocumentSelectors, correct?

I would appreciate a corresponding hint on that here in the docs.

Comment thread internal/languages/multilang/gen/main.go
Comment thread cmd/fastbelt/generate.go Outdated
Comment thread internal/generator/services_generator.go
Comment thread cmd/build.go
Comment thread internal/generator/lexer_generator.go Outdated
Comment thread lexer/doc.go Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's adhere the common pattern of naming this file doc.go.

//go:generate go run ./gen

can go to services.go, like in the other languages.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open, correct?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Forgot about this one, changing it before merging 👍

@msujew
msujew force-pushed the msujew/multi-language-support branch 2 times, most recently from 72f6899 to dd2fdc6 Compare September 28, 2026 12:18
@msujew
msujew force-pushed the msujew/multi-language-support branch from f9c1cd9 to a42044f Compare September 29, 2026 16:46

@Lotes Lotes left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, that you have also added the multi-file support!

Comment thread cmd/fastbelt/main.go
Comment thread internal/grammar/validator.go Outdated
Comment thread internal/grammar/validator_test.go
Comment thread lexer/modes.go
Comment thread internal/grammar/utils.go Outdated
Comment thread internal/generator/lexer_generator.go Outdated
Comment thread internal/generator/lexer_generator_test.go
@msujew
msujew force-pushed the msujew/multi-language-support branch from 9a07595 to a650095 Compare October 14, 2026 11:11
@msujew
msujew requested a review from Lotes October 19, 2026 12:33

@sailingKieler sailingKieler left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite impressive contributions Mark, great job!

However, the setup of the language selectors that sometimes are generated to services_gen.go and sometimes have been put to services.go caused quite some headache for me.
I've made a proposal inline to improve that situation.

Comment thread workspace/language_selector.go Outdated
Comment thread language_selector.go Outdated
Comment thread language_selector.go Outdated
Comment thread workspace/document_validator.go Outdated
Comment thread util/glob/glob_test.go Outdated

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still open, correct?

Comment thread internal/languages/multilang/gen/main.go
Comment thread examples/statemachine/services.go
Comment thread internal/generator/services_generator.go
Comment thread internal/grammar/services.go
@msujew
msujew force-pushed the msujew/multi-language-support branch from 1528f27 to 6c0e1a4 Compare October 22, 2026 09:37
^ Conflicts:
^	cmd/fastbelt/generate.go

^ Conflicts:
^	internal/grammar/utils.go
^	parser/completion_parser.go
@msujew
msujew force-pushed the msujew/multi-language-support branch from 6c0e1a4 to 3665b17 Compare October 22, 2026 10:46

@sailingKieler sailingKieler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks very good to me 🚀

(I assume you wanna keep the name mentioned in https://github.com/TypeFox/fastbelt/pull/130/changes#r3879863202?)

@msujew
msujew force-pushed the msujew/multi-language-support branch from 3665b17 to b8c7f63 Compare October 22, 2026 12:48
@msujew
msujew merged commit c450aae into main Oct 22, 2026
5 checks passed
@msujew
msujew deleted the msujew/multi-language-support branch October 22, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Improvement] Support multi-language setups

5 participants