Bur Droid *does* include CJK characters (in both Chinese and Japanese variants). So it is perfectly normal, that the Droid level in fontconfig, uses the CJK blocks available in Droid (in may have not worked cleanly in the past, that was a bug which is now fixed). That’s the desired behaviour both for Fedora and Google (Droid & Noto uostream).
https://gitlab.freedesktop.org/fontconfig/fontconfig/-/issues/33

What it should not do is fallback from Noto to Droid, given that Noto also includes CJK blocks (descendants of Droid BTW). So your problem is not that the CJK resolution in Droid works. Your problem is that Noto does not include the same level of fontconfig unification right now.

Hmm, well, that's true. but google-noto-fonts doesn't have unified families even. and google-noto-cjk-fonts is a separate source package. this is yet a bit challenge for us. could you stop providing a fallback for Noto in google-droid-fonts, at least until it is done?

That looks a bit agressive to me. honestly I don't think Droid has such quality as a fall back.

> That looks a bit agressive to me. honestly I don't think Droid has such quality as a fall back.

Why? Noto is effectively Droid v2. It is the same design, produced by the same people, for the same use cases. There are some metric differences, but mostly, it is an evolution of the same family. The only reason it is not called Droid still is that Google realized the Droid naming was a mistake (for trademark reasons, and due to how they contracted out Droid to Ascender in the first place).

Droid is definitely the best Fedora font to fallback to if something asks for Noto and the corresponding Noto part is not available.

> Hmm, well, that's true. but google-noto-fonts doesn't have unified families even. and google-noto-cjk-fonts is a separate source package. this is yet a bit challenge for us. could you stop providing a fallback for Noto in google-droid-fonts, at least until it is done?

Is there an ETA for this? Droid has been unified in Fedora for a decade+. Noto upstream formally requested Noto unification. I don’t see how you can argue in this issue tracker, that unification is a challenge, while arguing in the fontconfig issue tracker, that the existing fontconfig syntax needs no changes and serves font deployment needs well.

Forgive me but this has been going on for 10+ years. The “at least until it is done” card expired long ago. Do you have a more definite forward plan than that?

> Why? Noto is effectively Droid v2. It is the same design, produced by the same people, for the same use cases. There are some metric differences, but mostly, it is an evolution of the same family. The only reason it is not called Droid still is that Google realized the Droid naming was a mistake (for trademark reasons, and due to how they contracted out Droid to Ascender in the first place).

If looking at Latin glyphs only, that may be true. that's not what I'm talking about. my concern is about Droid blahblahblah unified as *Droid*. those had never been considered to be worth changing default fonts for certain languages despite that other Noto families did for some. if you don't want to drop those lines, those poor quality fonts should be dropped from unified Droid. those aren't worth a fallback font as *unified Noto*.

> Is there an ETA for this? Droid has been unified in Fedora for a decade+. Noto upstream formally requested Noto unification. I don’t see how you can argue in this issue tracker, that unification is a challenge, while arguing in the fontconfig issue tracker, that the existing fontconfig syntax needs no changes and serves font deployment needs well.

On your efforts, new fonts packaging guidelines has taken effect. thank you for that. all the fonts packagers are encouraged to follow on it in f33 IMHO. probably good to have Change proposal for that? dunno.

> Forgive me but this has been going on for 10+ years. The “at least until it is done” card expired long ago. Do you have a more definite forward plan than that?

It's for you right. but not for most of people, because there were no strong reasons to do so and is really new in guidelines.

Anyway, this has been reported to *f32*. I don't think we had enough time to work on it. seeing packages followed by new guidelines was really nice and is really helpful to understand how we need to do. but having changes breaking other packages and pushing it without taking care of them are not a good idea, particularly for an upcoming stable release.

(In reply to Akira TAGOH from comment #4)
> > Why? Noto is effectively Droid v2. It is the same design, produced by the same people, for the same use cases. There are some metric differences, but mostly, it is an evolution of the same family. The only reason it is not called Droid still is that Google realized the Droid naming was a mistake (for trademark reasons, and due to how they contracted out Droid to Ascender in the first place).
> 
> If looking at Latin glyphs only, that may be true.

It’s not "looking only at the latin glyphs". Noto and Droid are both wide coverage font families, and Noto started by inheriting all the coverage already achieved in Droid. Google wide coverage efforts did not start with Noto. The latest Droid refresh (all the painful digging in the multi-gig Android source tree) was requested by people using Droid as a fallback font.

> that's not what I'm
> talking about. my concern is about Droid blahblahblah unified as *Droid*.
> those had never been considered to be worth changing default fonts for
> certain languages despite that other Noto families did for some.

You’re mixing things up. This is not a "default font" family situation. The user constructed a font stack that includes Noto. When Noto coverage is insufficient, fontconfig fallbacks to the closest font, which is Droid (as it should be, for design and historical reasons). All of this is fine and normal and works as it should.

What does not work as it should is that fontconfig ignores the CJK parts of Noto before fallbacking from Noto to another font. That‘s the problem root cause.

 if you
> don't want to drop those lines, those poor quality fonts should be dropped
> from unified Droid. those aren't worth a fallback font as *unified Noto*.

Again, Droid is not configured as a generic fallback font here.

> > Is there an ETA for this? Droid has been unified in Fedora for a decade+. Noto upstream formally requested Noto unification. I don’t see how you can argue in this issue tracker, that unification is a challenge, while arguing in the fontconfig issue tracker, that the existing fontconfig syntax needs no changes and serves font deployment needs well.
> 
> On your efforts, new fonts packaging guidelines has taken effect. thank you
> for that. all the fonts packagers are encouraged to follow on it in f33
> IMHO. probably good to have Change proposal for that? dunno.
> 
> > Forgive me but this has been going on for 10+ years. The “at least until it is done” card expired long ago. Do you have a more definite forward plan than that?
> 
> It's for you right. but not for most of people, because there were no strong
> reasons to do so and is really new in guidelines.

But that’s not what I wrote about. The problem is triggered by lack of Noto unification in fontconfig.

*I* did not make Google request unification. *I* did not make Adobe unify font in its products. *I* did not go to the ISO standardise unifications syntaxes. That‘s a huge amount of work by lots of people and companies. That’s not my own private project. That should be enough by itself to show “strong reasons” exist (and BTW, Adobe unification efforts were driven by the needs of CJK, not Latin users).

There are multiple, years old requests to make unification easier in fontconfig. Your answer so far has been that no syntax changes are needed, and font deployers can use the existing syntax just fine.

Well, here unification is needed to fix a user problem. Just do it then. That will solve the requester problem.

Alternatively, can we see a serious commitment to a more user-friendly syntax fontconfig-side? I’ve no problem to stage packaging changes on a shared roadmap. But, absent this roadmap, I won’t delay fixing my stuff for illusory “laters”.

(In reply to Nicolas Mailhot from comment #5)
> It’s not "looking only at the latin glyphs". Noto and Droid are both wide
> coverage font families, and Noto started by inheriting all the coverage
> already achieved in Droid. Google wide coverage efforts did not start with
> Noto. The latest Droid refresh (all the painful digging in the multi-gig
> Android source tree) was requested by people using Droid as a fallback font.

Let's say more specific thing then. as this report complains about CJK, Google Noto CJK fonts isn't even inherited from Google Droid Japanese nor Google Droid Fallback. the unification you did in Droid says "it is a part of Droid" now. but CJK support in Droid isn't worth a fallback for Noto.

> You’re mixing things up. This is not a "default font" family situation. The
> user constructed a font stack that includes Noto. When Noto coverage is
> insufficient, fontconfig fallbacks to the closest font, which is Droid (as
> it should be, for design and historical reasons). All of this is fine and
> normal and works as it should.

In fact your changes affected to the default font. not mixing things up.

> What does not work as it should is that fontconfig ignores the CJK parts of
> Noto before fallbacking from Noto to another font. That‘s the problem root
> cause.

Not exactly. from the point of view how fontconfig select the best font, current situation is coming from the misconfiguation in those rules in packages. so exactly speaking the root cause is:

>                                    The problem is triggered by lack of Noto
> unification in fontconfig.

That. As I said in my initial comment, we should update google-noto-fonts and google-noto-cjk-fonts for this issue. but introducing the change in google-droid-fonts without those changes as well looks agressive. particularly doing it in f32. in fact that breaks and affects as this report. we should release all the relevant packages together to avoid breakage, particularly for f32. that's all what I want to say.

> Alternatively, can we see a serious commitment to a more user-friendly
> syntax fontconfig-side? I’ve no problem to stage packaging changes on a
> shared roadmap. But, absent this roadmap, I won’t delay fixing my stuff for
> illusory “laters”.

Leaping of the logic. why such configuration was written by users is also same root cause. introducing the unification of Noto fixes all the problem here. no need to make changes in syntax. that is nice-to-have but not necessarily required to fix this.

(In reply to Akira TAGOH from comment #6)

> Let's say more specific thing then. as this report complains about CJK,
> Google Noto CJK fonts isn't even inherited from Google Droid Japanese nor
> Google Droid Fallback. the unification you did in Droid says "it is a part
> of Droid" now. but CJK support in Droid isn't worth a fallback for Noto.

Again, I disagree, for design and historical and user expectation reasons Droid as a whole is the best fallback for Noto as a whole. The problem here is not the fallback, the problem here is that fontconfig fallbacks for Noto when the CJK parts of Noto are available on system.

> > You’re mixing things up. This is not a "default font" family situation. The
> > user constructed a font stack that includes Noto. When Noto coverage is
> > insufficient, fontconfig fallbacks to the closest font, which is Droid (as
> > it should be, for design and historical reasons). All of this is fine and
> > normal and works as it should.
> 
> In fact your changes affected to the default font. not mixing things up.

They did not affect the default font. They affected a user-specific configuration. A user-specific configuration, BTW, that was attempting a limited Noto unification, because unification is the natural thing for users to do (as people have been reporting to fontconfig for years now).

You should be explaining to the user how to unify Noto and Noto CJK properly, since that is clearly the user wish, instead of trying to avoid the issue by breaking the configuration of other font families.

(I write "you" here because you’ve been repeatedly dismissing reports that fontconfig syntax is inadequate for use by normal users in presence of modern font families spread over multiple files. So here is the perfect opportunity to prove us all wrong. Or to change your mind and plan a fix. Either way works for me.)

> > What does not work as it should is that fontconfig ignores the CJK parts of
> > Noto before fallbacking from Noto to another font. That‘s the problem root
> > cause.
> 
> Not exactly. from the point of view how fontconfig select the best font,
> current situation is coming from the misconfiguation in those rules in
> packages. so exactly speaking the root cause is:
> 
> >                                    The problem is triggered by lack of Noto
> > unification in fontconfig.
> 
> That. As I said in my initial comment, we should update google-noto-fonts
> and google-noto-cjk-fonts for this issue. but introducing the change in
> google-droid-fonts without those changes as well looks agressive.
> particularly doing it in f32.

After 10+ years of heads-up upstream, with multiple problem reports, by multiple entities, it is not aggressive. It is way late.

>  no need to make changes in syntax. that is nice-to-have but not necessarily required to fix this.

Then just make use of this syntax yourself. Document and explain this syntax to users yourself. And do it for all the font families that need it (hint: that means all the wide coverage font families produced for mobile and web today, since they are *all* sub-setted and split for performance reason. Noto is no more an exception than Droid was. It’s the model font makers are following for all new font families).

Do not ask others to use a syntax that “needs no changes” if you do not want to touch it yourself.

Also the “let’s kludge things for « special family » so fontconfig can avoid looking at the general problem” is exactly what we’ve been doing for Droid for more than a decade now.

So, complaining about the consequences, while asking to repeat the approach for another font family, is more than a bit rich.

(In reply to Nicolas Mailhot from comment #7)
> (In reply to Akira TAGOH from comment #6)
> 
> > Let's say more specific thing then. as this report complains about CJK,
> > Google Noto CJK fonts isn't even inherited from Google Droid Japanese nor
> > Google Droid Fallback. the unification you did in Droid says "it is a part
> > of Droid" now. but CJK support in Droid isn't worth a fallback for Noto.
> 
> Again, I disagree, for design and historical and user expectation reasons
> Droid as a whole is the best fallback for Noto as a whole. The problem here
> is not the fallback, the problem here is that fontconfig fallbacks for Noto
> when the CJK parts of Noto are available on system.

So I'm wondering why you did push your packages only even though you know you break it. and I'm saying, if you want to push this to f32, all the required changes should be pushed together. the sort of this changes shouldn't be made in coming stable release.

> They did not affect the default font. They affected a user-specific
> configuration. A user-specific configuration, BTW, that was attempting a
> limited Noto unification, because unification is the natural thing for users
> to do (as people have been reporting to fontconfig for years now).

I'm sorry, but it does. Noto fallback rule in google-droid-fonts breaks CJK default fonts when both google-droid-sans-fonts and google-noto-sans-fonts are installed regardless of custom configuration, because the logic of the issue happening here is that "Droid Sans" is added after "Noto Sans", which means "Droid Japanese" and "Droid Fallback" also has priority next to "Noto Sans". this is quite bad.

This isn't custom configuration related, but "Droid sans overrides my default CJK font" as summary says.

> Do not ask others to use a syntax that “needs no changes” if you do not want
> to touch it yourself.

my bad. seems missing some wording. but I'm not saying "need no changes" totally. we need it but not necessarily to fix *this issue*. sounds off topic to me though, or was it a response on it?

> So I'm wondering why you did push your packages only even though you know you break it.

First, this change did not break any Fedora package or config. It broke one *user* configuration.

And second, I’ve been telling fontconfig upstream for years the syntax in fontconfig was not clean and robust enough for today’s needs (today as in the today of 12 years ago, yes the reports have been going that long). At one point, I bowed to your expertise and pushed stuff with the existing syntax.

So, not going back after waiting 12 years if that means you can keep ignoring the need. Which you *still* are in all your replies.



The problem is not in the way Droid was unified. That has been requested (and not just by me) for 12+ years.



The problem is not that Noto is not unified. Sure it needs unification like Droid did but IT IS NOT THE ONLY WIDE COVERAGE FONT FAMILY OUT THERE. There are lots of them, Google and Adobe and IBM and others have been funding their creating for a decade now.



Please propose a way forward that makes it easy for the requester, or any other fontconfig user, to unify the wide coverage font families they want to use. Because neither me, nor any other Fedora packagers, are going to unify the whole mass of font files that exist out there.


FEDORA PACKAGERS CAN NOT SUBSTITUTE FOR USER-FRIENDLY FONTCONFIG SYNTAX.

Sorry for late response, I missed the notification mail from bugzilla.

(In reply to Nicolas Mailhot from comment #10)
> > So I'm wondering why you did push your packages only even though you know you break it.
> 
> First, this change did not break any Fedora package or config. It broke one
> *user* configuration.

As I said in the previous comment, this is irrelevant to users configuration. this issue can be reproduced without custom configuration once you install both google-droid-sans-fonts and google-noto-sans-cjk-*-fonts.

> And second, I’ve been telling fontconfig upstream for years the syntax in
> fontconfig was not clean and robust enough for today’s needs (today as in
> the today of 12 years ago, yes the reports have been going that long). At
> one point, I bowed to your expertise and pushed stuff with the existing
> syntax.

I'm sorry but I don't understand why you started to talk about the syntax here...
We are talking about the issue we are facing *NOW* which is actually a regression.

BTW I'm seeing another problem about font family unification. but it is off-topic for this. I'll bring it up on the list shortly.

(In reply to Akira TAGOH from comment #11)
> As I said in the previous comment, this is irrelevant to users
> configuration. this issue can be reproduced without custom configuration
> once you install both google-droid-sans-fonts and
> google-noto-sans-cjk-*-fonts.

One correction. google-droid-sans-fonts and google-noto-sans-cjk-*-fonts /and/ google-noto-sans-fonts to be reproduced.

Only google-noto-sans-fonts in them are not installed by default. google-droid-sans-fonts are most likely to be installed by default because of deps from ghostscript.

Akira, what is the best action here?

(In reply to Jens Petersen from comment #13)
> Akira, what is the best action here?

There is two things we need to do. one is to have a config in google-noto-*cjk-*fonts to make them a part of Noto like Droid Fallback is a part of Droid. I would keep this bug for that.
Another one is to change the priority number in config file for google-droid-sans-fonts to the higher number than google-noto-fonts. i.e. 67 or later one, because google-noto-sans-fonts use 66.

Leter one simply avoids a situation where fontconfig choose Droid as the best font rather than Noto when both are available.

I agree it makes sense to lower google-droid-fonts priority, since it is not a default user font, but installed by default due to gs.

In light of https://fedoraproject.org/wiki/Changes/DefaultToNotoFonts, Noto Sans is going to be installed by default in F36 and it seems this issue will become more visible (website requesting Noto Sans, etc) even without user-specific configuration. Would this be reconsidered?

I wonder if this is the correct place to post my issue,
but I'm somehow been hit by this issue here.
Though this might perhaps be a different issue.

I'm on an uptodate Fedora 35

$ dnf list installed | grep noto
google-noto-cjk-fonts.noarch               20201206-3.fc35                     @fedora               
google-noto-cjk-fonts-common.noarch        20201206-3.fc35                     @fedora               
google-noto-emoji-color-fonts.noarch       20200916-3.fc35                     @fedora               
google-noto-fonts-common.noarch            20201206-3.fc35                     @fedora               
google-noto-sans-cjk-ttc-fonts.noarch      20201206-3.fc35                     @fedora               
google-noto-sans-fonts.noarch              20201206-3.fc35                     @fedora               
google-noto-serif-cjk-ttc-fonts.noarch     20201206-3.fc35                     @fedora  

$ dnf list installed | grep droid
google-droid-sans-fonts.noarch             20200215-10.fc35                    @fedora   

The droid font is a dependency used by e.g. gimp and atril. so I'm not be able to deinstall it.

What's annoying, is, that if I request Noto with a font fallback chain, chinese characters get substituted with the droid font

$ pango-view --font="Noto Sans,Noto Sans SC Regular,Noto Sans CJK SC Regular,Noto Sans CJK SC, 16" --no-display --output=out1-droid-installed.desktop.fc35.png --serialize-to=ser1-droid-installed.desktop.fc35.json utf-8.txt
$ pango-view --font="Noto Sans CJK SC, 16" --no-display --output=out2-droid-installed.desktop.fc35.png --serialize-to=ser2-droid-installed.desktop.fc35.json utf-8.txt

as can be seen in the ser1-*-installed*.json file the chinese symbols come from the Droid font.
where as in the ser2.json file there is no mentioning of the Droid font at all.

$ dnf remove google-droid-sans-fonts.noarch

$ pango-view --font="Noto Sans,Noto Sans SC Regular,Noto Sans CJK SC Regular,Noto Sans CJK SC, 16" --no-display --output=out1-droid-notinstalled.desktop.fc35.png --serialize-to=ser1-droid-notinstalled.desktop.fc35.json utf-8.txt
$ pango-view --font="Noto Sans CJK SC, 16" --no-display --output=out2-droid-notinstalled.desktop.fc35.png --serialize-to=ser2-droid-notinstalled.desktop.fc35.json utf-8.txt

as can be seen now in the ser1-*-notinstalled*.json file the chinese symbols come from the Noto font.

I didn't expect such a behavior, Noto fonts are installed, Noto fonts are requested, but for chinese symbols Droid symbols are delivered.

Kind regards
Andre

Created attachment 1850357 [details]
archive of pango-view output

Hi Andre, I think the pango-view tool doesn't support the font fallback list.

FEDORA-2022-568c77bfd5 has been submitted as an update to Fedora 36. https://bodhi.fedoraproject.org/updates/FEDORA-2022-568c77bfd5

FEDORA-2022-568c77bfd5 has been pushed to the Fedora 36 stable repository.
If problem still persists, please make note of it in this bug report.