In T117845#5306809, @Fomafix wrote:

In T117845#5175196, @Joe wrote:

1. I see no community discussion approving this change across multiple wikis in the task description. I only see a (partial) technical discussion above

Was there a community discussion for introducing the not BCP 47 conform language codes sr-ec and sr-el instead of the BCP 47 conform language codes sr-cyrl and sr-latn? The standard BCP 47 is not made by the community.

No of course not, as the website literally predates the new fangled standard (2009) as well as any sort of 'process' being in place to begin with.

2. Changing a whole set of urls like this has effect on incoming links, or search engine ranking, and quite a few other things. This is one of the reasons the old adage "cool urls don't change" still holds valid

Yes, "cool urls don't change". But cool URLs can redirect to a new place with a 301 Moved Permanently. It is also possible to keep the current URLs and add new URLs that redirect the to current URLs. It is not intended to break any links or rankings. The URL attributes already support the BCP 47 conform language codes: https://www.mediawiki.org/?uselang=sr-latn

Hmm, true, but is it needed ? I don't know, something like that should definitely spun into a separate ticket if you ask me.

3. I see no product decision behind the change, which means not all the angles and consequences have been thought out.

HTML must use BCP 47 conform language codes for several values. MediaWiki should use standard conform language codes to avoid the forward and backward mapping. The product MediaWiki should use standard conform language codes because MediaWiki is also used in not WMF related projects. WMF-related projects may keep the own language codes at places like URLs.

HTML and urls are not the same thing. There is no reason a url is required to conform to using bcp-47 language codes. While we do tend to adhere to bcp-47 language codes going forward for new site urls, that's another thing from swapping urls on existing properties.

In T117845#5306821, @TheDJ wrote:

No of course not, as the website literally predates the new fangled standard (2009) as well as any sort of 'process' being in place to begin with.

BCP 47 is much older than 2009. sr-ec has even with RFC 1766 from 1995 the meaning Serbian in Ecuador.

Hmm, true, but is it needed ? I don't know, something like that should definitely spun into a separate ticket if you ask me.

The aim of this task is to use the BCP 47 conform language codes as internal language codes in MediaWiki. This simplifies MediaWiki because the forward and backward mapping from external to internal and back to external language codes is superfluous. The current internal should still supported at all places where backward compatibility is needed for example in URLs. It is also possible to generate for WMF projects the old URLs. This still much less mapping because there are much less places where URLs are generated than places where HTML is generated in the code.

HTML and urls are not the same thing. There is no reason a url is required to conform to using bcp-47 language codes. While we do tend to adhere to bcp-47 language codes going forward for new site urls, that's another thing from swapping urls on existing properties.

Of course URLs can contain any value. But it does not harm when the URLs (also) support BCP 47 conform language codes.

Change 467770 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/core@master] Step 1 of renaming sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/467770

Change 251312 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/core@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/251312

Change 520999 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[operations/mediawiki-config@master] Map the variant URLs /sr-cyrl/ and /sr-latn/ to /sr-ec/ and /sr-el/

https://gerrit.wikimedia.org/r/520999

Change 521000 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/InputBox@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/521000

Change 521024 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/InputBox@master] Rename language codes 'sr-ec' and 'sr-el' to 'sr-cyrl' and 'sr-latn'

https://gerrit.wikimedia.org/r/521024

Change 323677 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/Echo@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323677

Change 323681 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/LiquidThreads@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323681

Change 323678 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/FlaggedRevs@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323678

Change 323684 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/Thanks@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323684

Change 323680 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/Gadgets@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323680

Change 287141 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/Wikibase@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/287141

Change 323683 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/ParserFunctions@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323683

Change 323667 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/CategoryTree@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323667

Change 521053 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/Wikibase@master] Restore tests with language codes 'sr-cyrl' and 'sr-latn'

https://gerrit.wikimedia.org/r/521053

Change 522144 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[translatewiki@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522144

Change 522186 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/UniversalLanguageSelector@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522186

Change 522354 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/MobileFrontend@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522354

Change 522379 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[mediawiki/extensions/ExtJSBase@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522379

Change 522379 merged by Robert Vogel:
[mediawiki/extensions/ExtJSBase@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522379

Change 368248 had a related patch set uploaded (by Fomafix; owner: Fomafix):
[operations/puppet@production] Add additional aliases for sr-cyrl and sr-latn next to sr-ec and sr-el

https://gerrit.wikimedia.org/r/368248

As part of T174601: Change the language codes of Sakizaya from "ais" (retired by SIL) to "szy" everywhere, and add it to Names.php we now have tooling to move messages from one language code to another, both in translatewiki.net and in the repositories.

This is not sufficient however. Great deal should be taken to setup proper language fallbacks, making sure the variant URLs still work, being clear what language codes will be used, figuring out migrations in WikiData, etc.

This all should be explored and have a plan written in the task summary. Feedback should be requested from relevant parties, it might be worth to go through RfC or similar process to make sure that it happens.

Change 522144 abandoned by Nikerabbit:
[translatewiki@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

Reason:
This requires a migration plan.

https://gerrit.wikimedia.org/r/522144

Is there a way to define sr-cyrl and sr-latn as alias for sr-ec and sr-el ?

Change 521053 abandoned by Addshore:

[mediawiki/extensions/Wikibase@master] Restore tests with language codes 'sr-cyrl' and 'sr-latn'

Reason:

Abandoning all Wikibase.git patches that have not been touched since 2019.
If you want to revive this patch, or keep it around for your reference please re open it!
We just got to below 100 open changes for Wikibase.git!

https://gerrit.wikimedia.org/r/521053

Change 287141 abandoned by Addshore:

[mediawiki/extensions/Wikibase@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

Reason:

Abandoning all Wikibase.git patches that have not been touched since 2019.
If you want to revive this patch, or keep it around for your reference please re open it!
We just got to below 100 open changes for Wikibase.git!

https://gerrit.wikimedia.org/r/287141

Change 375616 had a related patch set uploaded (by Fomafix; author: Fomafix):

[operations/mediawiki-config@master] Add language codes sr-cyrl and sr-latn next to sr-ec and sr-el

https://gerrit.wikimedia.org/r/375616

Change 522354 abandoned by Jdlrobson:

[mediawiki/extensions/MobileFrontend@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

Reason:

Clearing out the review queue for 2022. Please restore if the core patch gets renewed traction.

https://gerrit.wikimedia.org/r/522354

Change 790357 had a related patch set uploaded (by Fomafix; author: Fomafix):

[operations/deployment-charts@master] Add additional aliases for sr-cyrl and sr-latn next to sr-ec and sr-el

https://gerrit.wikimedia.org/r/790357

Change 467770 merged by jenkins-bot:

[mediawiki/core@master] Step 1 of renaming sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/467770

Change 522354 restored by Fomafix:

[mediawiki/extensions/MobileFrontend@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522354

Change 287141 restored by Fomafix:

[mediawiki/extensions/Wikibase@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/287141

Change 816224 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/extensions/Cite@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/816224

Change 824729 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/core@master] Add 'sr-cyrl' and 'sr-latn' to Names.php

https://gerrit.wikimedia.org/r/824729

Change 521053 restored by Fomafix:

[mediawiki/extensions/Wikibase@master] Restore tests with language codes 'sr-cyrl' and 'sr-latn'

https://gerrit.wikimedia.org/r/521053

Change 827520 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/extensions/Translate@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/827520

Change 828511 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/core@master] Remove support for messages with the language code sr-ec and sr-el

https://gerrit.wikimedia.org/r/828511

Change 829259 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/core@master] Rename sr-ec.json to sr-cyrl.json and sr-el.json to sr-latn.json

https://gerrit.wikimedia.org/r/829259

Change 830658 had a related patch set uploaded (by Fomafix; author: Fomafix):

[mediawiki/core@master] Support sr-cyrl and sr-latn in language converter

https://gerrit.wikimedia.org/r/830658

+1 for language code to conform syntax of language, script, region codes to BCP 47--script codes can be in 4-letter Alpha in sentence form or numeric; region codes in 2-letter ALPHA in capital letters.

Examples for Chinese language (Mandarin, Min Nan and Cantonese (Yue), scripts in Traditional and Romanized scripts.
zh-nan-Hant-TW (Chinese spoken in Min Nan, written in Traditional script in Taiwan)
zh-Hant-HK (Chinese written in Traditional script in Hong Kong, assuming in Mandarin language)
yue-HK (Chinese spoken in Cantonese in Hong Kong)
cmn-Latn (Chinese Mandarin written in Romanized form, (not necessarily Pinyin), ie. a Romanization for Qianlong Emperor: "Tchien Lung Whang Tee" (from a 1797 publication)

validator: https://schneegans.de/lv/

Thank you.

zh-nan-Hant-TW (Chinese spoken in Min Nan, written in Traditional script in Taiwan)

Based on the IETF BCP 47 language tag specification, it's recommended to use nan-Hant-TW instead of zh-nan-Hant-TW (nan-* instead of zh-nan-*, same for yue-Hant-HK vs. zh-yue-Hant-HK) .

Perfect. Thanks for the updates.

Change 521000 merged by jenkins-bot:

[mediawiki/extensions/InputBox@master] Rename language codes sr-ec and sr-el to sr-Cyrl and sr-Latn in tests

https://gerrit.wikimedia.org/r/521000

I think we were waiting for TranslateWiki to rename the codes? How is that going?

Is there a task for that? It's not in our radar currently. We haven't renamed language codes in many years, so the process is unclear. We have some docs in https://translatewiki.net/wiki/Renaming_language_codes

Change 421793 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/421793

In T117845#9200743, @gerritbot wrote:

Change 421793 merged by jenkins-bot:

[mediawiki/extensions/cldr@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/421793

I assume removing sr-ec and sr-el here is why sr-ec and sr-el no longer have an English name - T348366

Change 323667 merged by jenkins-bot:

[mediawiki/extensions/CategoryTree@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323667

In T117845#9290941, @gerritbot wrote:

Change 323667 merged by jenkins-bot:

[mediawiki/extensions/CategoryTree@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323667

Translation pages have been renamed on translatewiki.net .

In T117845#9291616, @Winston_Sung wrote:

In T117845#9290941, @gerritbot wrote:

Change 323667 merged by jenkins-bot:

[mediawiki/extensions/CategoryTree@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323667

Translation pages have been renamed on translatewiki.net .

Change also shoud be made on Translatewiki, messages are still made with old codes by default

In T117845#9291680, @Milicevic01 wrote:

Change also should be made on translatewiki, messages are still made with old codes by default.

I'm afraid that there's no way to partially disable language codes by projects on translatewiki for now.

In T117845#9291755, @Winston_Sung wrote:

I'm afraid that there's no way to partially disable language codes by projects on translatewiki for now.

Any thoughts about this? Like disabling sr-* language codes on translatewiki until complete the JSON file renaming process (but I think that won't be a good idea, though).

@Nikerabbit
@jhsoby

In T117845#9291755, @Winston_Sung wrote:

I'm afraid that there's no way to partially disable language codes by projects on translatewiki for now.

Actually, there is:

$wgTranslateDisabledTargetLanguages = [
	'*' => [...], // languages disabled in all groups
	'ext-categorytree' => ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
	'ext-categorytree-api' =>  ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
	'ext-categorytree-user' => ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
];

By the way, there is no i18n system for these messages, but I’d probably include both English and Serbian text in these strings to make them easier to understand for Serbian translators.

Actually, there is:

$wgTranslateDisabledTargetLanguages = [
	'*' => [...], // languages disabled in all groups
	'ext-categorytree' => ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
	'ext-categorytree-api' =>  ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
	'ext-categorytree-user' => ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
];

By the way, there is no i18n system for these messages, but I’d probably include both English and Serbian text in these strings to make them easier to understand for Serbian translators.

Serbian version: Молимо преводите на sr-cyrl. & Molimo prevodite na sr-latn.
Also messages translated through Special:Translate shoud be generated with new codes automatically
I hope i helped
Regards

In T117845#9293923, @Tacsipacsi wrote:

Actually, there is:

$wgTranslateDisabledTargetLanguages = [
	'*' => [...], // languages disabled in all groups
	'ext-categorytree' => ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
	'ext-categorytree-api' =>  ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
	'ext-categorytree-user' => ['sr-el' => 'Translate to sr-latn please.', 'sr-ec' => 'Translate to sr-cyrl please.'],
];

Oh, good to know.

How would it work?

• ['*'] + ['ext-categorytree'] for 'ext-categorytree' => how to enable sr-cyrl and sr-latn ?
• ['ext-categorytree'] for 'ext-categorytree' => duplicate other disabled language codes?

In T117845#9294156, @Milicevic01 wrote:

By the way, there is no i18n system for these messages, but I’d probably include both English and Serbian text in these strings to make them easier to understand for Serbian translators.

Yeah, pretty sad.

In T117845#9295157, @Winston_Sung wrote:

How would it work?

• ['*'] + ['ext-categorytree'] for 'ext-categorytree' => how to enable sr-cyrl and sr-latn ?

Oh, I didn’t realize these cannot be translated to. It turns out that although they’re not disabled (disabled languages are defined here: rGTWN mw-config/TranslateSettings.php:163-227 (at 11774909d725)), MediaWiki core “helpfully” translates sr-cyrl to sr-ec and sr-latn to sr-el, making them unavailable in Special:Translate because it normalizes language codes using the LanguageFactory:

$languageFactory = MediaWiki\MediaWikiServices::getInstance()->getLanguageFactory();
$languageFactory->getLanguage( 'sr-cyrl' )->getCode() === 'sr-ec'
$languageFactory->getLanguage( 'sr-latn' )->getCode() === 'sr-el'

Editing a single message, on the other hand, doesn’t do such normalization, and thus seems to work (I successfully saved a null edit; I didn’t want to experiment with real changes). Truly disabled languages, like als, give a permission error when trying to edit a single message.

• ['ext-categorytree'] for 'ext-categorytree' => duplicate other disabled language codes?

There is no need to duplicate other language codes. Languages can be disabled at three levels:

• Globally, on all of translatewiki.net
• For a given root message group (the root is the part before the first - in the message group ID)
• For a given single message group

The different levels are merged automatically. So while ext-categorytree, ext-categorytree-api and ext-categorytree-user all need to be specified because ext-categorytree doesn’t count as a root message group, languages disabled at a global or root ext level don’t need to be repeated.

In T117845#9294156, @Milicevic01 wrote:

Serbian version: Молимо преводите на sr-cyrl. & Molimo prevodite na sr-latn.

Thanks!

We got complaints from translatewiki.net: https://translatewiki.net/wiki/Support#c-Milicevic01-20231212102100-MediaWiki:Cldr-desc/sr-cyrl

Why was https://gerrit.wikimedia.org/r/c/mediawiki/extensions/cldr/+/421793 merged? It seems premature to me.

See https://codesearch.wmcloud.org/search/?q=%5C%2Bes-419&files=&excludeFiles=&repos= for the preferred way of configuring languages per group. But note that sr-cyrl is not enabled on translatewiki.net so nobody can translate using that code.

When will this be fully implemented and available on Wikipedia, for instance? Will URLs also be affected? I hope so. I'm asking because of another project that uses Wikidata as its source, which, I believe, still uses sr-ec and sr-el.

Change #522354 abandoned by Jdlrobson:

[mediawiki/extensions/MobileFrontend@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

Reason:

Hello this is an automated message.
I am abandoning this patch as it over a year old, and is not currently in a mergeable state. This has nothing to do with the quality of the patch.

If you still care about this patch, please feel free to restore it and rebase it, and we can happily continue the conversation to help you get it merged.

https://gerrit.wikimedia.org/r/522354

Change #522354 restored by Fomafix:

[mediawiki/extensions/MobileFrontend@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522354

Change #481318 merged by jenkins-bot:

[mediawiki/extensions/WikimediaMessages@master] Step 1 of renaming sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/481318

Change #522354 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/522354

Change #323683 merged by jenkins-bot:

[mediawiki/extensions/ParserFunctions@master] Rename language codes sr-ec and sr-el to sr-cyrl and sr-latn

https://gerrit.wikimedia.org/r/323683

Change #1051680 had a related patch set uploaded (by Winston Sung; author: Winston Sung):

[mediawiki/core@master] Update jquery.i18n from 1.0.7 to 1.0.10

https://gerrit.wikimedia.org/r/1051680

Change #1051680 merged by jenkins-bot:

[mediawiki/core@master] Update jquery.i18n from 1.0.7 to 1.0.10

https://gerrit.wikimedia.org/r/1051680