Despite being a huge proponent of a shared code and template repository, I think this proposal has one fundamental flaw, and might need to be rethought before implementing. It presumes and encourages two types of content: shared for everyone and used by a single wiki. Yet, I believe the single-wiki override will not be much different than the current situation, and will only result in more fragmentation. Instead, I think we should implement a shared repository without any options to override. This way if a subset of wikis need different content, they will simply create a copy with a slightly different name. We will still encourage cross-wiki content reuse, and each wiki can either go with version A or version B (or C, ...) of the content, yet all will be easily accessible from the same location.
If the only reason shadow namespaces were introduced was to not create new namespaces, I think we should at least prohibit creation of templates or modules if identically named page already exists in the shared repo. This will be similar to how we merged user accounts. Eventually more and more pages will be merged in, making wikis more consistent.

This feature will really help a lot to the ones who maintain wiki-farms or have wikis on separate subdomains as multi-language solution.

@Yurik, do you propose to restrict creating of particular templates/modules in wikis, but fetch them from shared repository? This sounds like a very strange idea.

A potential problem I foresee is different languages wanting to use the same page name. If wikis A, B and C all want to use page name X, then which wiki's content is used?

@tom29739 nah, doubt its a major issue - if the same title is needed by multiple languages (which by definition is unlikely), they can use longer titles qualifying the difference, e.g. Parameters and Parameters RTL
@Vedmaka, I don't argue with the how great it would be to share cross-wiki. I am concerned about allowing per-wiki override.

Shadow namespaces attempt to create OOP-style inheritance model, allowing selective override of features. But I feel this is a mistake in this case because one would have to override the entire module or template, not a specific piece of it. Thus it is still a per-wiki copy, only with an extra unknown. If we keep all same modules/templates on the same wiki, searchable and usage-trackable, it would be greatly simplify things.

The only time where we really do need per-language customization is when we do translations. But that should not be done as shadows, instead I think we should use mw.loadData() from subpages - allowing translate wiki to be setup to update it all in one place. Especially because sometimes we shouldn't localize based on the source wiki, but rather based on the user's preferences.

@Yurik, I agree and changed my own version in Module:Central.fr.ws and my last short description of task.

In T91162#2408661, @Yurik wrote:

Shadow namespaces attempt to create OOP-style inheritance model, allowing selective override of features. But I feel this is a mistake in this case because one would have to override the entire module or template, not a specific piece of it. Thus it is still a per-wiki copy, only with an extra unknown. If we keep all same modules/templates on the same wiki, searchable and usage-trackable, it would be greatly simplify things.

I don't think you have to override the entire module or template. We could use labeled section transclusion for overriding specific parts of pages. or have a central lua module that exposes functions, and local wikis can augment or hook into those functions. The reason for using page as the as the "unit" of cross-wiki content copying is because it is generally the most reasonable unit that already exists...

In T91162#2596074, @Legoktm wrote:

... We could use labeled section transclusion for overriding specific parts of pages. or have a central lua module that exposes functions, and local wikis can augment or hook into those functions. The reason for using page as the as the "unit" of cross-wiki content copying is because it is generally the most reasonable unit that already exists...

I agree that page is ideal as a unit, but this will also mean that most of the time this entire unit will be copied. Could you give an example of a usecase where a subsections would be used? Also, I'm worried about cross-wiki managing all such pieces - I would much rather have them all together.

In my Module:Central which defines the basic libraries and functions of central modules I already "augment or hook some functions" in modules with different names without ambiguity.
In cases of identical modules names that could disturb the process if different cascades of modules change functions definitions in unpredictible ways, like in well known multiple inheriting in C++.

@Legoktm, would this be a good topic for this week's IRC meeting (E269)?

I don't think there's anything much to talk about that we didn't discuss last time? I'm still working on implementing what was discussed at the last meeting.

@Legoktm, is there a summary of what exactly you are implementing in this step?

In T91162#2408647, @tom29739 wrote:

A potential problem I foresee is different languages wanting to use the same page name. If wikis A, B and C all want to use page name X, then which wiki's content is used?

I believe the goal currently is to replicate/emulate the behavior of global user pages and foreign file repositories: if the local page/file exists, prefer that, otherwise fall back to the global/centralized version.

In T91162#2408633, @Yurik wrote:

Instead, I think we should implement a shared repository without any options to override. This way if a subset of wikis need different content, they will simply create a copy with a slightly different name. We will still encourage cross-wiki content reuse, and each wiki can either go with version A or version B (or C, ...) of the content, yet all will be easily accessible from the same location.

How do you envision your proposed implementation being deployed? It sounds like you'd want to flip a switch and suddenly pages would start inheriting from a central wiki, even if the local page exists. This sounds like a very difficult migration path, it would be inconsistent with existing behavior for global user pages and files, and it would probably be a pretty confusing user experience for everyone.

If the only reason shadow namespaces were introduced was to not create new namespaces, I think we should at least prohibit creation of templates or modules if identically named page already exists in the shared repo.

Users want flexibility, of course. I don't need to convince you of that. :-) For global user pages, we already have some warnings in the user interface when editing a local user page of the same name. Same with local file description pages, at least on the English Wikipedia. We also have a Special page or some other database report that lists shadowed file names (i.e., file names that exist both locally and on the central wiki).

As a compromise, one could encourage people to delete the local versions once they have been imported into the central repository.

In any case, local users need to know if the available object is local or central. Perhaps from the style of the title and from a dedicated sign.

If the central version has priority, the local user cannot delete or rename this local object, and cannot choose which object use. Also the local user cannot rename and re-use it with another version name. Also that need an option to manage all cases.

In T91162#2718090, @Rical wrote:

In any case, local users need to know if the available object is local or central. Perhaps from the style of the title and from a dedicated sign.

Sure. We already have user-visible notices for files that are coming from a foreign file repository such as Wikimedia Commons. We also already have user-visible notices for global user pages coming from Meta-Wiki.

If the central version has priority, the local user cannot delete or rename this local object, and cannot choose which object use. Also the local user cannot rename and re-use it with another version name. Also that need an option to manage all cases.

While I agree with what you're saying, I think this is essentially the cost of having a centralized repository. You give up some local control over an item in exchange for having fewer versions/editions of a similar global item. (I'm reminded of Brexit!) We make this trade-off in a couple of places already, such as in the File and User namespaces. We want to generalize the functionality to make it easier to extend to other namespaces such as Help, as I understand it.

OK for central modules which have the priority and replace local ones.
I see 2 ways to retreave local versions:

• The replacement is registered in the history of the object page. Then any user can find and reuse the object before the central replace.
• Admins could have a right for this case.

@ahroni has an un conference session scheduled for today. Please sign up.

I added a session to the 2017 Dev Summit, which is closely related to this task. Hope to see some of this task's subscribers there. Sorry about the super-short notice, I'll do my best to make it useful nevertheless.

I started an Etherpad for the session: https://etherpad.wikimedia.org/p/devsummit17-xwiki-templates

People who understand the technology inside Shadow namespaces are welcome to add it.

On the enwiki, templates are created, put in use, merged and deleted all the time. It may be more complex to manage them on a shared repository for other projects, but "no real way" is simply wrong.

In T91162#2928782, @JEumerus wrote:

On the enwiki, templates are created, put in use, merged and deleted all the time.

Of course, and the same is true for each single wiki.

It may be more complex to manage them on a shared repository for other projects, but "no real way" is simply wrong.

"No real way" refers to sharing templates between wikis. And there is no real way for that. There is currently no shared repository, and there's no way to write a template on one wiki and use it on another, except importing it and adapting it manually, which creates a fork and doesn't get automatic updates from the source template.

Note-taker(s) of this session: Follow the instructions here: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Session_Guidelines#NOTE-TAKER.28S.29 After the session, DO NOT FORGET to copy the relevant notes and summary into a new wiki page following the template here: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Your_Session and also link this from the All Session Notes page: https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/All_Session_Notes. The EtherPad links are also now linked from the Schedule page (https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Schedule) for you!

Notes from this session: https://etherpad.wikimedia.org/p/devsummit17-xwiki-templates

I moved the notes to https://www.mediawiki.org/wiki/Wikimedia_Developer_Summit/2017/Cross-wiki_templates,_translatable,_reusable,_manageable

Thanks a lot for this, @Legoktm. I plan to follow up further.

Does it have to be "templates"? What if I want to transclude a poem from Wikisource into a Wikipedia article about the poem? Or a quotation into a Wiktionary entry, to show how a word gets used?

More information: https://www.mediawiki.org/w/index.php?title=Topic:T6ukmc98uuun7ydc (especially @Billinghurst's comment)

What's the current status of the Shadow namespace work? Has it gotten stalled?

This RFC seems to be stalled. If there is currently no interest in driving this further, it should for now be removed from the RFC work board.
If there is interest in continuing the RFC process, please let us (TechCom) known who will be working on this RFC, and who commits to implementing it if approved, and in what time frame.

@Fjalapeno "shadow namespaces" seems like a bit of infrastructure that audiences may be asking for. Is this a topic for ATWG?

@daniel potentially… there is definitely a desire to be able to reliably parse information from main pages, news portals, features article pages, etc…

Global templates may help this. But so may storing data via MCR.

I'd say that the topic of "making content machine readable and easier to parse" is a high level goal worthy of discussion.

@Fjalapeno Shadow namespaces don't really do anything for easier parsing, they are unrelated to MCR. This is about cross-wiki access to page content. Global templates would be one application of shadow namespaces. Global Lua modules and global Gadgets would be others.

Reading would like more mobile-friendly and possibly more machine-readable pages, and since most complex markup and most data that machines would care about is in templates, that would mainly be achievable by template standardization. (Putting machine-readable data into an MCR slot instead of the current mark-it-up-in-HTML-then-parse-out approach would be cool, but seems quite far away, and in any case would still happen through templates in some form.) Cross-wiki template standardization and the creation of a specialist class users focusing on that (much like the Commons community for images or Wikidatans for data) would increase the chance of that happening (in the case of most small wikis, increase it from zero) and that depends on shadow namespaces. So there is definitely a product need IMO.

Also global gadgets might or might not be implemented via shadow namepsaces and since frontend developers are negatively affected by broken / misbehaving gadgets that interfere with the code they are working on, anything that would improve average gadget quality is also a win.

That said, I thought the main driver for this was CommTech (with global gadgets & templates the #3 wish in 2015 and global gadgets the #1 in 2016)?

In T91162#3891228, @daniel wrote:

@Fjalapeno Shadow namespaces don't really do anything for easier parsing,

Sorry… probably need some context: Global templates, made possible by shadow namespaces, make it easier for parsing as they can be used to enforce markup consistency across wikis. So if the same templates are used across all news portals on all wikis, then we can use them to apply the same markup to the content. This is of course brittle, but it is a technique used by several services maintained by Readers - they parse content from main pages and news portals by looking at HTML tags. This makes it hard to roll out to every project, and we add new languages on a case by case basis. So this is the case I was talking about

they are unrelated to MCR.

Yes it is unrelated to MCR, but MCR is a better way to serve the use case I am talking about. Because at the end of the day, parsing this content is brittle, even with global templates. What would be better, is if pages could store the data that is needed in a structured way so that the services do not have to parse the page HTML to get the most recent news story. Instead they can just look in an MCR slot to get data that represents the news story.

Hopefully this makes more sense.

@Fjalapeno Ah, now I see the connection. So Reading's main interest in this is in standardizing templates, to make it easier to parse information out of wikitext and/or HTML? It seems to me T114251: [RFC] Magic Infobox implementation would be more relevant for that use case.

Most of the information is not infobox related (e.g. news entries or DYK entries on the main page). IMO the nice solution would be T156876: Structured data side channel for wikitext. But in some ways it's always going to rely on templates.

There is also the whole responsive templates thing (use TemplateStyles to add responsive CSS to templates which reacts meaningfully to screen width, instead of inline desktop-only CSS) which is an independent issue.

@brion: Should this task still be assigned to you? I think not, given that the concept of "shepherd" has gone away now?

Here's how I remember this task: I believe @Legoktm started working toward making a general shadow namespaces implementation so that it could be used with file description pages from a central wiki like Commons and global user pages from a central wiki like Meta-Wiki. When we discussed this during an IRC meeting, Tim suggested that templates and Scribunto modules would be a different beast than file and user pages (and maybe help pages?). Where we left off was that we would do the "easier" part first (File, User, and Help pages) and then figure out how to deal with the more complex part (Template and Module pages).

I think any work toward resolving this task has stalled now.

In T91162#5065305, @daniel wrote:

I agree with this analysis: sharing any "active" content (like templates, modules, user scripts, gadgets, etc) across wikis is going to be more tricky.

Not only more tricky but potentially requiring a different approach. For code you want code reviews, CI infrastructure and so on. Reinventing all that in MediaWiki might not be the best approach.

(Templates are more borderline - not quite code, but not quite content either.)

For one thing, we presently have no ability to track such cross-wiki usage.

We do for images, via the GlobalUsage extension. That approach could probably be extended to other things.

In T91162#5065385, @Tgr wrote:

In T91162#5065305, @daniel wrote:

For one thing, we presently have no ability to track such cross-wiki usage.

We do for images, via the GlobalUsage extension. That approach could probably be extended to other things.

Indeed. That feature is also (ab)used for user scripts (see T35355):
https://commons.wikimedia.org/wiki/Special:WhatLinksHere/Template:GlobalJsUsage