Use BreakIterator for grapheme-cluster-aware width when Mode 2027 is unavailable (JDK 21+)

Problem

When Mode 2027 is not available (terminal doesn't support it, or dumb terminal), all width calculations fall back to per-codepoint WCWidth.wcwidth() summing. This produces incorrect widths for multi-codepoint emoji sequences:

Sequence	wcwidth sum	Actual terminal width
👨‍👩‍👧 (family ZWJ)	2+0+2+0+2 = 6	2
🇩🇪 (flag)	2+2 = 4	2
👋🏽 (wave + skin tone)	2+2 = 4	2
🏳️‍🌈 (rainbow flag)	1+0+0+2 = 3	2

This affects both application-level width functions (columnLength(), columnSubSequence()) and Display.update() screen diffing, since Display uses the same code path internally. The result is misaligned borders, wrong padding, and render artifacts (ghost characters, cursor drift) — with no way for the application to fix it, because even if the application calculates correct widths, Display still uses the broken path.

Proposal

Since JDK 20, java.text.BreakIterator.getCharacterInstance() implements full UAX #29 Extended Grapheme Cluster segmentation (JDK-8291660). It correctly groups ZWJ sequences, regional indicator pairs, skin tone modifiers, keycap sequences, and tag sequences into single clusters.

JDK 21 added Character.isEmoji() and Character.isEmojiPresentation() for reliable emoji detection. Together with BreakIterator, this provides correct emoji width without any custom tables.

The idea: on JDK 21+, when Mode 2027 is not active, replace WCWidth.wcwidth() summing entirely with BreakIterator-based grapheme clustering:

Segment text into grapheme clusters using BreakIterator.getCharacterInstance()
Multi-codepoint cluster where first codepoint has Character.isEmoji() == true → width 2
Single codepoint with Character.isEmojiPresentation() == true → width 2
Single codepoint with Character.isIdeographic() == true → width 2
Everything else → width 1

This would fix width calculation in AttributedCharSequence.columnLength() (no-arg and terminal-aware-without-Mode-2027 paths), columnSubSequence(), and crucially in Display.update() — all without requiring Mode 2027 terminal support.

JDK compatibility

I understand JLine 4 targets Java 11+ and cannot require JDK 21. A runtime check could gate the entire approach:

private static final boolean HAS_GRAPHEME_CLUSTER_SUPPORT = 
    Runtime.version().feature() >= 21;

JDK 21+: use the BreakIterator + Character.isEmoji() path, no WCWidth.wcwidth() needed
JDK < 21: fall back to existing WCWidth.wcwidth() summing (current behavior, unchanged)

This preserves full backward compatibility while eliminating the broken wcwidth path on modern JDKs. The JDK 21 APIs (isEmoji, isEmojiPresentation, isIdeographic) combined with BreakIterator cover all width cases: emoji, CJK, fullwidth forms, and combining characters.

Context

I maintain charm.clj, a Clojure TUI library built on JLine. We prototyped this approach and confirmed that BreakIterator produces correct widths for all tested emoji categories without a terminal. However, applying it only at the application level doesn't help because Display still uses the wcwidth path internally — the fix needs to be in JLine itself to be effective.

fix: use grapheme cluster width on JDK 21+ without Mode 2027 (fixes #…

fix: use grapheme cluster width for no-terminal path on JDK 21+ (fixes …

feat: add cursor position probe fallback for grapheme cluster detection

fix: address CodeRabbit review feedback on cursor probe

fix: add comment on best-effort cleanup in cursor probe

fix: suppress SonarQube Thread.sleep warning in test helper

refactor: reuse CursorSupport.getCursorPosition for cursor probe

@TimoFreiberg

This has been addressed in PR #1754 with a three-tier detection strategy:

1. Mode 2027 (DECRQM) — existing path

Unchanged. Terminals that support Mode 2027 get grapheme cluster mode via CSI ? 2027 h/l.

2. Cursor position probe — new fallback

When Mode 2027 is not supported but the terminal responded to the DECRQM/DA1 query, JLine writes a test emoji (👋🏽) and queries cursor position via DSR/CPR. If the cursor advances by 2 columns, grapheme cluster mode is enabled without Mode 2027 escapes.

3. JDK 21+ BreakIterator — new fallback (as proposed in this issue)

On JDK 21+, WCWidth.charCountForDisplay() and wcwidthForDisplay() always use BreakIterator-based grapheme cluster segmentation, regardless of whether the terminal supports Mode 2027 or the cursor probe succeeded. This is the approach proposed by @TimoFreiberg and it fixes width computation in columnLength(), columnSubSequence(), columnSplitLength(), and by extension Display.update().

System property

org.jline.terminal.graphemeCluster can be set to true (force-enable, skip probing), false (disable), or left unset (auto-detect).

Known limitation: Apple Terminal flag emoji

Apple Terminal has a rendering quirk where regional indicator pairs (flag emoji like 🇫🇷) are rendered with a 2-cell-wide glyph but the cursor is only advanced by 1 cell. This causes the next character to be positioned under the flag glyph (hidden behind it). This is an Apple Terminal bug — JLine and the JDK both correctly compute width 2 for the flag, but Apple Terminal only advances the cursor by 1 after rendering it. All other emoji render correctly.

Easy way to reproduce on Apple Terminal:

printf '\r\e[K\U0001F1EB\U0001F1F712345678|\n'

Expected: 🇫🇷12345678| with all 8 digits visible. Actual: the 1 is hidden behind the flag glyph and the | is 1 column to the left. Compare with a simple emoji that works correctly:

printf '\r\e[K\U0001F38912345678|\n'

@TimoFreiberg — the JDK 21+ fallback should address your use case in charm.clj. The BreakIterator path now kicks in automatically on JDK 21+ without needing Mode 2027 terminal support.

Claude Code on behalf of Guillaume Nodet

fix: use grapheme cluster width for no-terminal path on JDK 21+ (fixes …

feat: add cursor position probe fallback for grapheme cluster detection

fix: address CodeRabbit review feedback on cursor probe

fix: add comment on best-effort cleanup in cursor probe

fix: suppress SonarQube Thread.sleep warning in test helper

feat: add cursor position probe fallback for grapheme cluster detection

fix: address CodeRabbit review feedback on cursor probe

fix: add comment on best-effort cleanup in cursor probe

fix: suppress SonarQube Thread.sleep warning in test helper

refactor: reuse CursorSupport.getCursorPosition for cursor probe

fix: grapheme-cluster-aware width when Mode 2027 is unavailable (fixes …

Update: PR #1754 now includes per-category emoji grouping detection.

What changed

The cursor probe has been extended from a single skin-tone emoji test to a 4-emoji batch probe — one per category:

Category	Test emoji	Example
`REGIONAL_INDICATOR`	🇫🇷	Flag pairs
`SKIN_TONE_MODIFIER`	👋🏽	Base + skin tone
`ZWJ_SEQUENCE`	👩‍🔬	Zero Width Joiner
`VS16_PRESENTATION`	☀️	Text + emoji presentation

Each category is detected independently by writing the emoji + DSR query in a single flush and checking if the cursor advanced by exactly 2 columns. This allows JLine to compute correct widths for terminals with partial emoji support.

Terminal results

Tested on macOS with 6 terminals:

Ghostty, iTerm2 — full Mode 2027 support, all emoji correct (no change from before)
Alacritty, Tabby, Superset — no Mode 2027, groups only REGIONAL_INDICATOR (flags). The 4-probe correctly detects this: flags get width=2, ungrouped emoji (ZWJ, skin tone, VS16) get per-codepoint widths matching cursor positions.
Apple Terminal — same as Alacritty for grouping detection. Apple Terminal's CPR is broken for complex ZWJ sequences (reports char count instead of column position), but this is an Apple Terminal bug that we don't try to work around — the visual rendering is still correct for all non-ZWJ emoji.

Width computation for ungrouped emoji

When a multi-codepoint emoji cluster is not grouped by the terminal, wcwidthUngrouped() sums individual codepoint widths with special handling for:

Skin tone modifiers (U+1F3FB–1F3FF): wcwidth=0 in combining table but render as width-2 standalone
Regional indicators (U+1F1E6–1F1FF): same treatment

New API

Terminal.EmojiGrouping enum with 4 categories
Terminal.getEmojiGroupings() returns the set of grouped categories

Claude Code on behalf of Guillaume Nodet

fix: grapheme-cluster-aware width when Mode 2027 is unavailable (fixes …

@gnodet

Thank you @gnodet !

Navigation Menu

Repository navigation

Use BreakIterator for grapheme-cluster-aware width when Mode 2027 is unavailable (JDK 21+) #1753

Problem

Proposal

JDK compatibility

Context

1. Mode 2027 (DECRQM) — existing path

2. Cursor position probe — new fallback

3. JDK 21+ BreakIterator — new fallback (as proposed in this issue)

System property

Known limitation: Apple Terminal flag emoji

2 remaining items

What changed

Terminal results

Width computation for ungrouped emoji

New API

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Use BreakIterator for grapheme-cluster-aware width when Mode 2027 is unavailable (JDK 21+) #1753

Description

Problem

Proposal

JDK compatibility

Context

Activity

gnodet commented on Apr 1, 2026

1. Mode 2027 (DECRQM) — existing path

2. Cursor position probe — new fallback

3. JDK 21+ BreakIterator — new fallback (as proposed in this issue)

System property

Known limitation: Apple Terminal flag emoji

2 remaining items

gnodet commented on Apr 1, 2026

What changed

Terminal results

Width computation for ungrouped emoji

New API

TimoKramer commented on Apr 7, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions