Support for Font Ligatures using Harfbuzz on Linux #2677

thunderseethe · 2019-07-26T01:41:21Z

Initial support for font ligatures using harfbuzz and free type on linux.
This builds on the work done by @anirudhb in [WIP] Basic text shaping and rendering using Harfbuzz
Step in the direction towards closing #50 .

This is relatively early in it's implementation but I wanted to get some eyes on it as changes past this point will start to effect core pieces of infrastructure (rendering etc.).

Currently code will render ligatures on linux with a monospace font (I've only tested with Fira Code).

Major Changes

TextRun replaces RenderableCell as main method of rendering text
Harfbuzz used to shape runs of text (allowing ligatures)
GlyphKey uses u32 (glyph index) instead of char

Support for font ligatures is enabled by changing the base rendering unit from a RenderableCell to a TextRun (name flexible). A TextRun bundles a series of RenderableCells that share the same render properties (line, fg, bg, etc.) into one unit. Each TextRun gets shaped by harfbuzz returning the glyph indices for each character. From here each glyph is rasterized as normal through the glyph cache.

Changes are guarded behind the hb-ft feature in alacritty_terminal.
Along with adding new code under this feature, some code is removed when this feature is turned on. Mainly code dealing with rendering RenderableCell's (and GlyphKey's c member).
The idea behind this is to ensure that there aren't any unknown uses of RenderableCell in the codebase. Once the code is free of RenderableCell when hb-ft is on, more intrusive changes can be made to rendering that break the conventions set by RenderableCell.

Caveats

Only works on linux.
Incurs performance hit of shape() call for each text run for each draw call (haven't done any testing yet to see how steep this is).
Ignores position information from harfbuzz, this means it probably only works for monospace ligatures (Fira Code) since each ligature is still rendered as if it's in it's own cell.
Each Face object carries an instance of hb_font with it, in addition to freetype::Face, this doesn't seem hugely problematic but is should definitely be noted.
Post shaping work is mainly done in Glyph Indices, this makes it hard to employ the same font recovery technique currently used by load_face_with_glyph as we don't have a char available to guess at the font. This functionality can be recovered by keeping the char stored with it's indice however this means GlyphKey would no longer fit in a u64.

Questions

Should TextRuns be cached similar to how GlyphKeys are currently?
Can this approach be adapted to Mac OSX and Windows, and if it can how different does the code need to be?
Currently code grabs a Vec<RenderableCell> from grid and wraps that as a TextRunIter to produce TextRuns. Would it make more sense for grid to directly produce a TextRunIter and avoid the unecessary middle allocation (hopefully)?
Should alacritty only support monospace ligatures and ignore "proper" ligatures?
Can TextRun completely replace RenderabelCell?
Does it make sense to use glyph index in GlyphKey (instead of char)?

Remaining Work

Rendering and Atlas construction should hopefully be able to take more advantage of TextRuns then they are currently (none).
Font ligatures should be a configurable option, this is a matter of enabling/disabling the harfbuzz feature based on a bool.
Harfbuzz supports building a font from a freetype Face directly, however none of the rust bindings expose that functionality currently.
Glyph Index is represented by a raw u32, this should be replaced with a newtype wrapper such as GlyphIndex(u32) similar to Line and Column.
Benchmarking performance with new TextRun implementation

I'm relatively new to the open source community so my apologies if something is out of place or I've missed a process I should've completed.

I initially ran my code through cargo fmt however it changed a ton of the codebase that I hadn't touched so I undid those changes. Is there a common rustfmt.toml config the repo uses?

Doesn't render properly yet, though.

CI and everything should now work.

… statements.

…t all share the same renderablecell except for (column, chars)

…hape one level deeper. Aim to move call into glyph_cache and expose through some API.

… method. Still not ideal as better use can probably be made of glyph_positions returned by harfbuzz but it will be involved to work those down to the Batch that renders cells at gl float values.

…inimal state needed.

…_run when hb-ft is on.

…for indice.

…used as glyph keys with hb-ft on. Clean up more extraneous changes.

…critty into font-ligatures

thunderseethe · 2019-12-03T17:12:30Z

I've just merged the winit/glutin rework PR which changed a lot of code, so that caused a few conflicts. If you need any help resolving these, please let me know.

My comment was in reference to @chrisduerr landing a large window rework in master that I needed to merge in this PR. I believe I have that merging done. However since this PR has been sitting stagnant there is still plenty of merging to be done if you'd like to take a look.

The next merge on my stack is to look at the new urls.rs file and add a method to it to work with TextRuns.

davidhewitt · 2019-12-03T23:56:24Z

Oh I see; you meant support for the updated windowing system, not the Windows OS! 😄

Once this PR lands I would be interested in attempting support for Windows. Perhaps using the allsorts crate (which might also work for macOS).

thunderseethe · 2019-12-12T22:39:36Z

I'm taking a look at the URL stuff wrt to TextRuns and wanted to ask, can a URL span a change in color/font/metadata (essentially span two or more text runs). At the very least it appears they can span multiple lines which means they can be at least two runs in that regard, but wanted to clarify the other semantics.

chrisduerr · 2019-12-12T22:54:44Z

I'm taking a look at the URL stuff wrt to TextRuns and wanted to ask, can a URL span a change in color/font/metadata (essentially span two or more text runs). At the very least it appears they can span multiple lines which means they can be at least two runs in that regard, but wanted to clarify the other semantics.

A URL is in the following format: https://github.com/jwilm/alacritty/blob/master/alacritty/src/url.rs#L18-L23

As you can see, the lines is a Vec<RenderLine>. Which means one URL can contain multiple lines and each line can have a different color. That's the main reason why there are multiple lines, because URLs are not interrupted when colors change.

Each of these RenderLines might also span multiple lines in the terminal.

svalaskevicius · 2020-01-25T21:26:52Z

had a quick go at merging master to this branch... tests pass, the terminal shows up and I can work in it, but have not tested much more: thunderseethe#1

angvp · 2020-01-29T18:55:51Z

I guess this branch needs rebase from master as I have lots of merge conflicts and in order to keep a clean history .. instead of merging master into this branch, good to know that way works tho.

roychoo · 2020-02-22T02:13:30Z

Hi, looking forward to this, wouldn’t mind helping but I am a total rust newbie. Can I ask a noob question, why only linux?

kchibisov · 2020-02-22T02:39:51Z

Due to use of hardbuzz library, which is used primary on linux, for macOS we should like use CoreText and for Windows DirectWrite.

Keating950 · 2020-03-10T20:47:52Z

Do people anticipate that this will be merged in the near future? If not, I'd be interested in packaging this branch for the AUR.

chrisduerr · 2020-03-10T20:54:46Z

Do people anticipate that this will be merged in the near future? If not, I'd be interested in packaging this branch for the AUR.

This branch is severely outdated and requires some significant work just to get it back up to date.

angvp · 2020-03-11T13:27:18Z

I am not a rust dev but I'm willing to volunteer to update the branch if the maintainer keeps MIA? ping @thunderseethe ?

blaggacao

Wonder how you consider @wezm 's library

blaggacao · 2020-06-13T20:57:20Z

alacritty_terminal/src/renderer/mod.rs

+    //}
+
+    // Since shaping is not available on Windows/Mac OSX, text run glyphs are rasterized one by one
+    // as characters


https://github.com/yeslogic/allsorts ?

We're open to it if the performance works well and it properly supports all necessary features on the platforms it could be used for.

However someone would actually have to do the work to try that out of course.

And please, avoid pinging folks without a reason for it. thx.

The reason is that Weslay, if I recall correctly, had offered working together with anyone who would want to pick this up.

yeah, and if anyone had a code with the mentioned lib and they had problems with it then maybe it might be something where pinging could make sense.

Or when trying to summon the wizards before the raid, but that arguably doesn't always work 😉

Hello all, I would be happy to answer any allsorts related queries. There should be no cross-platform issues — there are no OS dependencies except libc. We ship builds of Prince, which uses allsorts on at least Linux, macOS, Windows, and FreeBSD. I will admit the the API is currently quite low level, we hope to eventually add an easier to use higher-level API, but have not had a chance to do that yet.

Allsorts is certainly looking like a great solution for supporting all platforms at the same time. So the primary question would be how well it performs and how nicely the API integrates with Alacritty, but I think generally a lower level API should be a pretty good fit for Alacritty.

Since you're already around @wezm, I'd imagine you might be aware of how allsorts compares to harfbuzz in performance? Specifically for running relatively simple combination of characters like ligatures on short-ish strings.

I'd imagine you might be aware of how allsorts compares to harfbuzz in performance? Specifically for running relatively simple combination of characters like ligatures on short-ish strings.

We have not done a lot of comparison with Harfbuzz at this point as the primary focus was accurate shaping, and matching or exceeding the performance of our previous shaping engine. There is an open issue to explore this further yeslogic/allsorts#2

Whilst quite fast, it's probably safe to assume that Allsorts is not as fast as Harfbuzz, as Harfbuzz has been optimised for performance over many years. That is not to say we ignored performance though. We have profiled and optimised it in order to meet, and then exceed the performance of shaping engine it replaced in Prince.

blaggacao · 2020-06-13T21:31:52Z

@thunderseethe I made a patch between your branch and @zenixls2 's fork which is reported to work atop recent versions. Hope it helps:
zenixls2_patch.zip

Since his version is heavily inspired by this PR, the relevant diffs should be easy to consume. (hopefully)

Within the root of the workdir of your branch:
patch -p 1 -i zenixls2_patch.diff

(After that you should be able to squash all your commits and cleanly rebase onto current master without a single merge conflict... I assume here - since this is a WIP branch - the detailed commit history can be forgone.) Or just continue as if you where on current master...

Note: I haven't taken an involved look at the relevant diffs between your's and zneixls2's. Just thought this triangulation might save you some hours.

# both repos on ligature related branches
# get `.git` out of the way
mv .git ../
mv ../../zenixls2/alacritty/.git ../../zenixls2
git diff --no-index . ../../zenixls2/alacritty  >> zenixls2_patch.diff
# fix paths
sd "../../zenixls2/alacritty" "." zenixls2_patch.diff
# move git back
mv ../.git .
mv ../../zenixls2/.git ../../zenixls2/alacritty

blaggacao · 2020-06-13T22:07:36Z

I'm taking a look at the URL stuff wrt to TextRuns and wanted to ask, can a URL span a change in color/font/metadata (essentially span two or more text runs). At the very least it appears they can span multiple lines which means they can be at least two runs in that regard, but wanted to clarify the other semantics.

You (@thunderseethe) might want to have a look at this changeset:

zenixls2@126093a

felippemr · 2020-08-27T23:22:42Z

👋 , what are the things we need to do in order to merge this pr?

chrisduerr · 2020-08-28T10:19:49Z

@felippemr Check the previous comments please.

exorcist365

This is fucking awesome

Mic92 · 2021-03-27T18:45:27Z

Has anybody a newer rebase than zenixls2@126093a ?

carlosala · 2021-09-29T19:52:42Z

Hi! This seems to be quite near to be possible to merge. I am not rust dev, but I would use ligatures quite a lot.
If any dev is interested, it seems to work but this changes have to be upstreamed to actual repo status and revise the @chrisduerr changes.
Thanks!

This revives alacritty/alacritty#2677 changes specifically for the old `font` crate.

hack and others added 30 commits July 21, 2019 12:39

Add harfbuzz-freetype support

2d59156

Doesn't render properly yet, though.

Fix cargo.lock

ab620c9

CI and everything should now work.

Refine harfbuzz shaping more, now vaguely resembles the real text.

bc7d5e4

Fix clippy errors

1218fab

Break everything

a2ab0cc

Rendering is normal but font ligatures still don't work

f28f6fe

whitespace funky.

5df92b4

Tried setting scale and ppem but it didn't change anything.

79facbc

Switch to harfbuzz_rs and change GlyphKey to use a char or glyph_index

662fbe5

Looking into why whitespace is rendering is incorrectly

e1e33c8

Place holder commit to check something in master. Just more debugging…

a4084bc

… statements.

Replace sequence of maps with an iterator that produces text runs.

146f824

Created new TextRun struct to represent the idea of a run of text tha…

a3f8d65

…t all share the same renderablecell except for (column, chars)

Create render_text_run function on Renderer, this pushes call to hb s…

87df12b

…hape one level deeper. Aim to move call into glyph_cache and expose through some API.

cache now exposes shape method that calls underlying rasterizer shape…

a54de7b

… method. Still not ideal as better use can probably be made of glyph_positions returned by harfbuzz but it will be involved to work those down to the Batch that renders cells at gl float values.

Cleaned up opt_pair impl

03daff8

Remove cloning of rc.inner by introducing RunStart struct that hold m…

dea2a41

…inimal state needed.

Store string of rendered chars and zero width chars in text run.

09a6f9b

Replace render_string's calls to render_cell with call to render_text…

88f334d

…_run when hb-ft is on.

Clean up warning and remove some code that is no longer used.

09755da

Clean up some of the code that is no longer needed and swap GlyphKey …

7c3896b

…for indice.

Fix build with hb-ft feature turned off.

dd69ce6

Clean up superfluous changes from rustfmt that clutter PR.

cf777a1

Continuing clean up of PR.

d3513f3

Removed HbError and related code. Fixed errors were chars were being …

a32a835

…used as glyph keys with hb-ft on. Clean up more extraneous changes.

Added documentation to the text_run module.

de83c0b

Further clean up to reduce code in PR.

4bfedf0

Merge branch 'master' into font-ligatures

54c3c97

Fix build error in cursor.rs when hb-ft feautre is off

f1b5388

Merge branch 'font-ligatures' of https://github.com/thunderseethe/ala…

d970528

…critty into font-ligatures

davidhewitt mentioned this pull request Dec 10, 2019

Windows: Refactoring & cleanups #3085

Closed

5 tasks

chrisduerr added the C - waiting on author label Jan 15, 2020

roychoo mentioned this pull request Mar 14, 2020

Continue font ligatures #3456

Closed

davidhewitt mentioned this pull request May 16, 2020

Support for ligatures #50

Open

blaggacao reviewed Jun 13, 2020

View reviewed changes

This comment has been minimized.

Sign in to view

exorcist365 approved these changes Feb 9, 2021

View reviewed changes

This comment has been minimized.

Sign in to view

fee1-dead added a commit to fee1-dead/crossfont that referenced this pull request Nov 11, 2021

Add harfbuzz feature

beb71a2

This revives alacritty/alacritty#2677 changes specifically for the old `font` crate.

fee1-dead mentioned this pull request Nov 11, 2021

Add harfbuzz feature alacritty/crossfont#34

Closed

fee1-dead mentioned this pull request Dec 17, 2021

Support for Font Ligatures using harfbuzz #5696

Draft

chrisduerr closed this Dec 28, 2021

mikayla-maki mentioned this pull request Jul 26, 2022

Terminal zed-industries/zed#1211

Closed

64 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Support for Font Ligatures using Harfbuzz on Linux #2677

Support for Font Ligatures using Harfbuzz on Linux #2677

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

Support for Font Ligatures using Harfbuzz on Linux #2677

Support for Font Ligatures using Harfbuzz on Linux #2677

Uh oh!

Conversation

Uh oh!

Major Changes

Caveats

Questions

Remaining Work

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!