8000 The Return of the Embedded Font Issue · Issue #102 · prawnpdf/ttfunk · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

The Return of the Embedded Font Issue #102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
TastyPi opened this issue Mar 27, 2024 · 30 comments · May be fixed by #106
Open

The Return of the Embedded Font Issue #102

TastyPi opened this issue Mar 27, 2024 · 30 comments · May be fixed by #106

Comments

@TastyPi
Copy link
TastyPi commented Mar 27, 2024

1.8.0 has introduced issues with embedding fonts with prawn, see prawnpdf/prawn#1346

Minimal reproduction:

#!/usr/bin/env ruby

require "prawn"

pdf = Prawn::Document.new
pdf.font_families.update(
  "roboto" => {
    normal: "Roboto/Roboto-Regular.ttf",
    italic: "Roboto/Roboto-Italic.ttf",
    bold: "Roboto/Roboto-Bold.ttf",
    bold_italic: "Roboto/Roboto-BoldItalic.ttf",
  }
)
pdf.font("roboto")

pdf.text_box("€")

File.open("test.pdf", "w") do |file|
  pdf.render(file)
end

system "flatpak run com.adobe.Reader test.pdf"

The Euro symbol seems to trigger the bug.
@TastyPi TastyPi mentioned this issue Mar 27, 2024
Closed
@Salman464
Copy link
Salman464 commented Apr 4, 2024 •
edited
Loading

Encountered the same error recently Cannot extract the embedded font '3a9d5f+Recoleta-Bold'. Some characters may not display or print correctly.

Faced issue on the versions:

  • pdf-core (0.10.0)
  • prawn (2.5.0)
  • ttfunk (1.8.0

Resolution:
Downgraded gems to resolve:

  • pdf-core (0.9.0) (automatically downgraded with prawn and ttfunk)
  • prawn (2.4.0)
  • ttfunk (1.7.0)

I can confirm that downgrading to previous versions of prawn and ttfunk resolved the issue. not sure though if the issue is from ttfunk or prawn upgrade

@TastyPi
Copy link
Author
TastyPi commented Apr 4, 2024

@pointlessone something I forgot to mention is the PDF renders perfectly fine in Chrome and Evince, the error only appears in Adobe Reader. Which I imagine means the issue is extremely subtle and complicated 😅

@xtr3me
Copy link
xtr3me commented Apr 11, 2024

For me this causes CUPS not being able to print my PDF's over the IPP protocol. After reverting back to 1.7 everything works correctly again.

@ashkulz ashkulz mentioned this issue Apr 12, 2024
Open
@lucaong
Copy link
lucaong commented Apr 15, 2024

Same as what @xtr3me noted: we also have issues printing over CUPS, and reverting to 1.7 works.

If it helps, printing over CUPS gives the following error (which is printed by the printer):

ERROR:
invalidfont
OFFENDING COMMAND:
awidthshow

@jenskutilek
Copy link
Contributor
jenskutilek 8000 commented Apr 23, 2024 •
edited
Loading

I've ran a diff on an embedded font in a prawn pdf between ttfunk 1.7.0 and 1.8.0, and the only differences are in the maxp table.

Apparently the old version just kept the maxp values from the original font, which is a valid approach. The new version tries to recalculate them, but fails. I've added the value I'd consider correct in the rightmost column.

field                original  broken  corrected
maxPoints                 124       8         50
maxContours                 7       1          1
maxCompositePoints        150       0         50
maxCompositeContours        5       0          1
maxSizeOfInstructions    1028      61       1028
maxComponentElements        3       0          1
maxComponentDepth           1       0          1

maxSizeOfInstructions contains the length of the prep table's instruction bytecode for this font, because the prep table has the longest TrueType assembly program of all glyphs, and of the prep and fpgm tables.

There are several correct ways to calculate the value. It could also measure only the length of the longest glyph program (that's what the OpenType spec says). In that case, the result would be 133 in the subsetted font. The value 61 is wrong in any case.

fonts.zip

@yoshoku yoshoku mentioned this issue Apr 23, 2024
Open
@nishio-dens
Copy link

I am outputting PDFs using Japanese fonts, e.g. IPAFont. As reported here, I am having problems opening them in Acrobat Reader.

I am not familiar with ttfunk / maxptable / font implementations, so I may be thinking about the wrong thing, but I doubt that the code below is really correct.

The following process is described at line 110 of ttf_encoder.rb.
The argument is old_to_new_glyph.

ttfunk/lib/ttfunk/ttf_encoder.rb

Line 110 in 885061e

@maxp_table ||= TTFunk::Table::Maxp.encode(original.maximum_profile, old_to_new_glyph)

but, the receiver process appears to expect new_to_old_glyph.

ttfunk/lib/ttfunk/table/maxp.rb

Line 86 in 885061e

def encode(maxp, new2old_glyph)

Is the argument correct?

If this is not relevant, please ignore this comment.

@makiRails makiRails mentioned this issue Apr 23, 2024
Open
@drgcms
Copy link
drgcms commented May 22, 2024

Same problem here with ttfunk 1.8. It is required by prawn 2.5.0.

The funny thing is when opened in the browser pdf is displayed without special UTF-8 characters. But when printed, special characters are printed ok. This is all on Windows 11 with the last version of Acrobat Reader.

The previous version 8000 of the reader or Linux pdf readers are perfectly ok.

Reverting to ttfunk 1.7 also reverted pdf-core to 0.9.0 and prawn to 2.4.0 and it works as expected.

by
TheR

amstark612 added a commit to TinyMission/crossdoc that referenced this issue Jun 13, 2024
@amstark612
downgrade to prawn 2.4.0 so we can downgrade to ttfunk 1.7.0 prawnpdf…
7e5702f 
…/ttfunk#102
@bai-yi-bai
Copy link

I would also like to report unique behavior for PDFs generated using ttfunk 1.8.
Both stduviewer and DiffPDF foss v2.1.3 (GPL 2.0) are relatively old applications and exhibit similar behavior. They open files, but have lots of characters missing in them. I've also reverted to using ttfunk 1.7.

thibaudgg added a commit to csa-admin-org/csa-admin that referenced this issue Jun 26, 2024
@thibaudgg
Downgrade to Prawn 2.4.0 because of font issue
40e3e6b 
prawnpdf/ttfunk#102
@formigarafa
Copy link

Heads-up,
I noticed some missing lower case "g" on Open Sans after upgrading to versions mentioned above.
But instead of changing prawn or downgrade it I have downloaded a newer version of Open Sans and the 'g's are back.

Please note that I am not sure if there is any other glyph missing but in a superficial check over the list of all characters I could produce with the keyboard the all did appear in the reader. There was also a notification "cannot extract embedded font" which stop appearing.

This is the list of chars/glyphs I've used, I hope it helps anyone who may need something to start from.

ab a b
0123456789
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
`~!@#$%^&*()-_=+[]\{}|;':",./<>?
`´¨ˆ˜
¡™£¢∞§¶•ªº–≠œ∑®†¥øπ“‘«åß∂ƒ©˙∆˚¬…æΩ≈ç√∫µ≤≥÷
ŵèéêêëěẽēėę
ř
țťþ
ýŷÿ
ùúûüǔũūűů
ìíîïiǐĩīıį
òóôöǒœøõō
àáâäǎæãåāā
ßşșśš
ďð
ğġ
ħ
ķ
łļľ
źžż
çćčċ
ñńņň

@jasonperrone
Copy link

Heads-up, I noticed some missing lower case "g" on Open Sans after upgrading to versions mentioned above. But instead of changing prawn or downgrade it I have downloaded a newer version of Open Sans and the 'g's are back.

Please note that I am not sure if there is any other glyph missing but in a superficial check over the list of all characters I could produce with the keyboard the all did appear in the reader. There was also a notification "cannot extract embedded font" which stop appearing.

This is the list of chars/glyphs I've used, I hope it helps anyone who may need something to start from.

ab a b
0123456789
abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
`~!@#$%^&*()-_=+[]\{}|;':",./<>?
`´¨ˆ˜
¡™£¢∞§¶•ªº–≠œ∑®†¥øπ“‘«åß∂ƒ©˙∆˚¬…æΩ≈ç√∫µ≤≥÷
ŵèéêêëěẽēėę
ř
țťþ
ýŷÿ
ùúûüǔũūűů
ìíîïiǐĩīıį
òóôöǒœøõō
àáâäǎæãåāā
ßşșśš
ďð
ğġ
ħ
ķ
łļľ
źžż
çćčċ
ñńņň

Eager to try that, thank you.

@pointlessone
Copy link
Member

@jasonperrone This is an interesting observation. Could you please attach both the old "broken" and the new versions of the font?

@tobischo
Copy link
tobischo commented Jul 1, 2024

We experienced the same issue.
Attached you can find the older and newer font files we had in use here.

opensans-new.zip
opensans-old.zip

With the newer version + ttfunk 1.8 Adobe Acrobat is still throwing an error, but the letters seem to be shown correctly (the missing gs for example)
So for now, that still keeps us at ttfunk 1.7

@jasonperrone
Copy link
8000

I have also downgraded to prawn 2.4.0, ttfunk 1.7.0. I think that's all I can do. I downloaded the current versions of all the OpenSans-xxx.ttf fonts I use and they are all identical to the ones I already had.

@mojavelinux mojavelinux mentioned this issue Jul 3, 2024
Closed
@KjellMorgenstern
Copy link
KjellMorgenstern commented Sep 2, 2024 •
edited
Loading

Edit: TL;TR : Nothing new, except 'g' in bold OpenSans still missing and NotoSansThaiLooped is also affected (Bhat symbol).

Long text:
I also tried updating from some old OpenSans (copied from google fonts 2021) to some later Open Sans (downloaded from google fonts just today).
This fixed the missing "g" in regular text. However, in bold text, the "g" is still missing. Also, the "€" sign is missing.

I tried NotoSans instead. Now, only the € sign is missing.
Also, some currency symbols (Bhat) from the fallback font are missing.

I use pdf.fallback_fonts ["NotoSans", "NotoSansThaiLooped"]

In December 2023 I had added a comment to my code: # TODO We can't yet replace OpenSans with NotoSans. OpenSans has reduced line heights in the address sections. But with NotoSans, the default height seems to be used everywhere.
The height problem seems to be fixed, and OpenSans can be replaced with NotoSans.

Since the € symbol is still missing, I will now try the downgrade to prawn 2.4.

Acrobat Reader version is 2024.002.21005

@jenskutilek
Copy link
Contributor

It's not the fonts that are broken. The maxp table calculations in ttfunk are currently incorrect. Any given font or even different subsets of the same font may work or not work. The only way to fix it is to correct the calculations, or to go back to the previous code version, which didn't touch the values inside the maxp table when embedding a font.

@MrSerth MrSerth mentioned this issue Sep 18, 2024
Open
2 tasks
@wuarmin
Copy link
wuarmin commented Oct 8, 2024

Hey,
I came across the same issue. How should we continue here?
thanks

@flavio-b
Copy link
flavio-b commented Oct 14, 2024 •
edited
Loading

There's an open PR (#104 ) that attempts to fix the issue with the maxp.

@swistak
Copy link
swistak commented Oct 17, 2024

@wuarmin In the meantime lock those versions in Gemfile:

gem 'ttfunk', '~> 1.7.0'
gem 'prawn', '~> 2.4.0'

@samyouel
Copy link
samyouel commented Jan 7, 2025

This is an issue for us as well. Would be great to see this fixed! Thanks guys!

dacook added a commit to dacook/prawn that referenced this issue Mar 2, 2025
@dacook
Downgrade ttfunk
40ccf7c 
 Skips ttfunk 1.8 until issue prawnpdf/ttfunk#102 fixed
This was referenced Mar 2, 2025
Closed
Open
@wuarmin
Copy link
wuarmin commented Mar 24, 2025

Hey @pointlessone
how do we proceed here? I fear that staying with old versions will lead to problems at some point. prawnpdf is so great. It should go on.

thanks for your feedback!

@pointlessone
Copy link
Member

I mean, the plan is not complex: either someone contributes a fix or I claw out some time. I make no promises on my part at the moment.

I haven't looked into the issue but if my memory serves me right it's not maxp table. I think it's incorrect subsetting of the CFF table. It requires reindexing, or encoding remapping, or something. TBH, I don't remember which issue is this, there was a few very similar but actually different issues around subsetting.

One thing that is relatively easy you can help with is to go through the tickets, try repro scripts with both OTF and TTF fonts, and subsetting enabled and disabled (so 4 cases total). Post your discoveries in the relevant threads.

@jenskutilek
Copy link
Contributor

To quote myself:

Apparently the old version just kept the maxp values from the original font, which is a valid approach. The new version tries to recalculate them, but fails.

The easiest solution would be to revert the maxp subsetting to the previous behaviour. It seems that is what PR #104 does.

The maxp table contains values to help the renderer determine how much memory to reserve for different aspects of a font, so it isn't a problem when the values in the maxp table are "too large" (= matching the un-subsetted font).

@dacook
Copy link
dacook commented Mar 24, 2025

either someone contributes a fix or I claw out some time.

that is what PR #104 does.

So someone has contributed a fix (at least, a first step which makes the gem functional again), but you've mentioned a lack of time to merge it. It's not a big change to review, and a revert should be pretty acceptable for this situation.
But perhaps you need some testing to ensure it's a valid fix. So I guess someone could go ahead and test the PR with the four cases mentioned, and post their findings.

with both OTF and TTF fonts, and subsetting enabled and disabled (so 4 cases total)

Ideally, I think we want automated tests that show what broke, then it can prove how it was fixed. There's already automated tests (https://github.com/prawnpdf/ttfunk/tree/master/spec), and there's both an OTF and TTF test, so maybe these tests need expanding to cover the bug.

@pointlessone
Copy link
Member

@dacook Reviewing #104 is not that hard as it's a revert. Testing is the hard part. Fonts are literally half a century of cruft. What might work for one font file is not guaranteed to work with all or even most. Even if the thing is fully spec-compliant it still might not work in many cases. This makes automated testing very hard, too. You can't just read the spec and encode that. I mean, you should and that's a good first step but then you also have to test for compatibility with other software.

Anyway, since you spoke up would you please test the change with a bunch of fonts please?

@dacook
Copy link
dacook commented Mar 25, 2025

Sure, I'm not really familiar with fonts so wasn't sure how best to test, but scrolling back to the top I realise there's a minimal repro right there, so I will start with that when I get a chance tomorrow.

@dacook
Copy link
dacook commented Mar 26, 2025

I set up a test which is ready to feed fonts into: https://github.com/dacook/ttfunk-test
But forgot that I needed Adobe Reader to reproduce the problem.. will check again when I have time.

@pointlessone
Copy link
Member

Not only Adobe Acrobat but also Apple Preview, PDF.js, and something based on poppler (e.g. Evince). And also a printer because documents can render OK on screen but still be broken when printed.

@thatsmydoing thatsmydoing linked a pull request Mar 26, 2025 that will close this issue
Open
@thatsmydoing
Copy link
thatsmydoing commented Mar 26, 2025 •
edited
Loading

There is a bug where the wrong glyphs are used to calculate maxp. Edit: this was actually pointed out almost a year ago #102 (comment)

I've only tested for our small use case which doesn't involve any composite glyphs and it fixed it for me. I don't know if the rest of the maxp calculations are correct if there are composite glyphs.

@jenskutilek
Copy link
Contributor
jenskutilek commented Mar 26, 2025 •
edited
Loading

I set up a test which is ready to feed fonts into: https://github.com/dacook/ttfunk-test But forgot that I needed Adobe Reader to reproduce the problem.. will check again when I have time.

Thank you for taking the initiative!

As the problem lies not in the PDF embedding, but in the embedded font itself, it may be easier to check the validity of the modified font(s) directly.

For example, running the broken font from my first post (#102 (comment)) through the OpenType Sanitizer, I get this message related to the maxp table:

WARNING: glyf: Component depth exceeds maxp maxComponentDepth in glyph 2, adjust limit to 1.
WARNING: glyf: Number of composite points in glyph 2 exceeds maxp maxCompositePoints: 50 vs 0, adjusting limit.
WARNING: glyf: Number of contour points exceeds maxp maxPoints, adjusting limit (glyph 3)
WARNING: glyf: Bytecode length is bigger than maxp.maxSizeOfInstructions 61: 133 (glyph 3)
Failed to sanitize file!

Not sure why this is listed as a warning as it actually breaks the rendering of the font.

There's also the Font Validator which finds this:

Image

@mjansing
Copy link
mjansing commented May 5, 2025

Any updates on this? It would be great if there is a solution besides downgrading ttfunk.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Pass correct glyph mapping when encoding maxp thatsmydoing/ttfunk
0