[with patch] requiring a platform package that is provided by another required package fails #5030

dzuelke · 2016-03-09T19:00:48Z

First, this issue is not entirely trivial to reproduce, because everyone has different PHP versions and extensions installed. To reproduce the steps and behaviors below, please ensure that you run php -n $(which composer) instead of plainly composer, or that with composer show --platform | grep mongo, nothing shows up :)

If you're on Ubuntu or something, php -n will also prevent loading ext-phar, so nothing will work. In that case, just make sure you have no ext-mongo, and run composer directly instead of php -n $(which composer).

Let's say your code needs ext-mongo (via Doctrine ODM, for instance), but that's not available on PHP 7, so you want to use https://github.com/alcaeus/mongo-php-adapter, which in its composer.json says "provide": { "ext-mongo": "1.6.12" }, and which is a wrapper around ext-mongodb to provide an interface that's compatible with the old ext-mongo.

Start with a basic composer.json, to mock the existence of ext-mongodb (again, you do not want ext-mongo installed when following along, so always run with php -n later):

$ composer init -n
$ composer config platform.php "7.0.4"
$ composer config platform.ext-mhash "7.0.4"
$ composer config platform.ext-mongodb "1.1.3"

I will use the composer-provide branch of https://github.com/alcaeus/mongo-php-adapter explicitly because the master branch has changed between using provide and replace a few times recently. There is also a composer-replace branch that uses replace instead of provide for ext-mongo.

Remember, php -n will prevent ini loads, so if you have ext-mongo locally, it should not get loaded:

$ php -n $(which composer) require alcaeus/mongo-php-adapter:dev-composer-provide
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Installing mongodb/mongodb (1.0.1)
    Loading from cache

  - Installing alcaeus/mongo-php-adapter (dev-composer-provide 698d301)
    Cloning 698d3012c306af299491aa166dc9aa29a4bce5b8

Writing lock file
Generating autoload files

Now there is a package that provides "ext-mongo" installed, and that info is in the lock file. Which works as expected if you now require that extension:

$ php -n $(which composer) require ext-mongo:*
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Nothing to install or update
Writing lock file
Generating autoload files

A subsequent composer update also works. Great, so let's try a case that fails - having the requirements already in place, but no lock file:

$ rm -rf composer.lock vendor
$ php -n $(which composer) update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested PHP extension ext-mongo * is missing from your system. Install or enable PHP's mongo extension.

Weird, right? The same happens if you require both in one go obviously:

$ php -n $(which composer) remove ext-mongo
$ php -n $(which composer) remove alcaeus/mongo-php-adapter

$ php -n $(which composer) require alcaeus/mongo-php-adapter:dev-composer-provide ext-mongo:*
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - The requested PHP extension ext-mongo * is missing from your system. Install or enable PHP's mongo extension.

Installation failed, reverting ./composer.json to its original content.

You could of course also manually put both packages into composer.json and run composer update, same effect (that's actually what happened when we removed lock file and vendor dir earlier and ran an update).

For the steps above, instead of requiring ext-mongo, you can also use doctrine/mongodb, which in turn requires ext-mongo. Same behavior.

There is a workaround: if you put the "alcaeus/mongo-php-adapter" package into a "package" repo inside composer.json, things work. Take this composer.json:

{
    "repositories": [
        {
            "type": "vcs",
            "url": "https://github.com/alcaeus/mongo-php-adapter"
        }
    ],
    "require": {
        "alcaeus/mongo-php-adapter": "dev-composer-provide",
        "doctrine/mongodb": "^1.2"
    },
    "config": {
        "platform": {
            "php": "7.0.4",
            "ext-mhash": "7.0.4",
            "ext-mongodb": "1.1.3"
        }
    }
}

That works:

$ rm -rf composer.lock vendor
$ php -n $(which composer) update
Loading composer repositories with package information
Updating dependencies (including require-dev)     
…

This pointed to a problem with information from Packagist, and indeed, it's due to behavior in (and data available to) ComposerRepository.

There is no information from Packagist on provided platform packages such as ext-mongo, so without a lock file, the info that one of the other required packages provides ext-mongo is apparently not there, and that's why things fail.

Here is the fix:

diff --git a/src/Composer/Repository/ComposerRepository.php b/src/Composer/Repository/ComposerRepository.php
index 5bd4e8a..91d6d58 100644
--- a/src/Composer/Repository/ComposerRepository.php
+++ b/src/Composer/Repository/ComposerRepository.php
@@ -371,7 +371,9 @@ public function whatProvides(Pool $pool, $name)
                         // override provider with its alias so it can be expanded in the if block above
                         $this->providersByUid[$version['uid']] = $package;
                     } else {
-                        $this->providers[$name][$version['uid']] = $package;
+                        foreach($package->getNames() as $nameName) {
+                            $this->providers[$nameName][$version['uid']] = $package;
+                        }
                         $this->providersByUid[$version['uid']] = $package;
                     }

However, that feels like a band-aid treating a symptom of the real underlying issue, which is that Packagist does not have a list of provider packages for platform packages:

$ ls ~/.composer/cache/repo/https---packagist.org/provider-ext*
ls: .composer/cache/repo/https---packagist.org/provider-ext*: No such file or directory

/cc @alcaeus @holtkamp @stof

The text was updated successfully, but these errors were encountered:

dzuelke · 2016-03-09T19:15:26Z

It's important to understand that in the case of a file like .composer/cache/repo/https---packagist.org/provider-psr$log-implementation.json, the "provider" part in the filename is not related to the "provide" feature, but instead a way of "real Composer" repositories (such as Packagist) to split up their data into smaller chunks.

However, as (maybe an unintentional?) a side effect, this allows dependencies on packages provided by others.

Check out ComposerRepository::whatProvides(). That basically loads .composer/cache/repo/https---packagist.org/provider-psr$log-implementation.json containing a list of all packages that provide psr/log-implementation, so if someone/something requires that package, and then a list of all packages providing that package are available to the solver, and that's why the whole thing works.

At least that's how I understand it. @Seldaek just told me "yeah this might all be accidental" ;)

curry684 · 2016-03-15T23:21:47Z

I think the patch is fine and might as well be a PR.

stof · 2016-04-20T13:20:59Z

Check out ComposerRepository::whatProvides(). That basically loads .composer/cache/repo/https---packagist.org/provider-psr$log-implementation.json containing a list of all packages that provide psr/log-implementation, so if someone/something requires that package, and then a list of all packages providing that package are available to the solver, and that's why the whole thing works.

The fact that this file contains all packages providing it is necessary. Otherwise, composer would not be able to look for psr/log-implementation if it has not loaded the other packages already (and having to know who provides the package defeats the purpose of the feature). This would make the bugfix dependant on the order in which the loading of the alcaeus/mongo-php-adapter metadata and the call to ->whatProvides('ext-mongo') happen.

I think the right fix would be for Packagist to dump such files for any package referenced in provide and replace, even when they are platform packages, and then for composer to remove the code bypassing the search for platform packages.
The fact that registering a VCS repository fixes the issue is a good sign that this is the root cause: the VCS repository is always loading all the metadata instead of relying on server-side computation of provider names (something necessary for Packagist, due to the amount of metadata available in its repository)

dzuelke · 2016-04-22T03:32:43Z

Yeah I agree that this should be fixed on the Packagist side (I pointed that out at the end of my original report).

My second comment was just to point out that the "provider-…" files are not specifically for the "provide" feature, but for the "providers" functionality of Packagist type repos. In case anyone confused the two ;)

stof · 2016-05-10T09:03:50Z

@dzuelke would you mind opening a PR on the packagist repo to provide a fix ?

stof · 2016-05-10T10:38:59Z

My second comment was just to point out that the "provider-…" files are not specifically for the "provide" feature, but for the "providers" functionality of Packagist type repos. In case anyone confused the two

Except that these features are actually related. The way the repository protocol works is that the provider file for psr/log-implementation contains metadata of all packages providing this package on Packagist (be it because of their name or a provide rule).
The issue is that Packagist will not dump properly some of these provider files, because of https://github.com/composer/packagist/blob/e3cc5b3d6fec1d78d0088b2b9e33828801c0a583/src/Packagist/WebBundle/Package/SymlinkDumper.php#L229 which won't match for ext-mongo for instance.

8000

dzuelke · 2016-05-10T11:00:27Z

The term "provides" in a Composer type repository refers to "packages provided by that repo", not "packages provided by other packages", @stof. It just happens that this feature is used for the provide keyword functionality as well. At least that's what the docs at https://getcomposer.org/doc/05-repositories.md#provider-includes-and-providers-url seem to indicate; correct me if I am wrong, @Seldaek.

Seldaek · 2016-05-10T11:29:42Z

I just don't think we should fix this at the packagist level right now, because it'll create even more legacy to remove later on. We want to clean this up in 2.0 and fix the providers handling so that the files on packagist do not have to include all providers (which would shrink some of them quite substantially). /cc @naderman

stof · 2016-05-10T11:53:29Z

@Seldaek but then, how would composer know which files it should load to find psr/log-implementation ?

And as I said, fixing the issue on Packagist is just a matter of updating the regex above so that it does not exclude package names like ext-mbstring

Seldaek · 2016-05-10T11:56:17Z

Composer does not and can not know which package to load for psr/log-implementation.. You should require an implementation, and that implementation providing psr/log-implementatoin should be enough for Composer to figure things out. Right now that last part fails if we remove the providers on packagist, but that's what we want to fix. The ext-mbstring issue in practice isn't an actual problem right now except for @dzuelke but AFAIK he solved it, so let's please not waste time discussing this and create more legacy for something that will be removed anyway.

stof · 2016-05-10T12:07:23Z

@Seldaek the patch provided by @dzuelke depends on the order in which packages are loaded. So it does not fix the issue fully (and would lead to WTF debugging where changing the order of requirements would magically make things break or work)

Seldaek · 2016-05-10T12:08:56Z

Yeah I never suggested we fix it with that patch for 2.0. There's a bigger refactoring planned that would fix this.

stof · 2016-05-10T12:15:42Z

OK, so which fix should we use for Composer 1.x ? 2.0 is not yet a thing, and this issue affects any package providing a polyfill for an extension. So this affects the ext-mongo polyfill mentioned by @dzuelke above (which is not solved as the only way to fix it he found was to register the package repository explicitly rather relying on packagist, which is not something easy for people affected by this issue) but would also affect the symfony polyfill once they tell composer that they are polyfill (see symfony/polyfill#54) instead of forcing any library to stop requiring ext-mbstring explicitly when they want to be compatible with these polyfills.

dzuelke · 2016-05-10T14:47:54Z

It's also not an issue for me mind you, since deploys to Heroku require a composer.lock. It's just that someone stumbled over it when deploying to Heroku, and I decided to investigate.

It's an issue for any user of a polyfill like the ext-mongo or Symfony one, since you can't composer update without an error if there is no composer.lock present.

alcaeus · 2016-05-13T04:32:34Z

It's an issue for me, since I have to run around telling people that they need to add provide: { "ext-mongo": "..." } in their composer.json at root level in order to get this to work. Even if it's a workaround, I'd appreciate a fix here.

robfrawley · 2017-02-22T18:49:58Z

What is the status of this? Trying to use any library that polyfills extension behavior is impossibly difficult and complex to get right. It shouldn't be. Correct me if I am wrong, but it looks like this issue is continually being pushed to the next release...but this should be a "blocker" as it quite literally describes a broken implementation for a feature that is documented involving the most important aspect of Composer: dependency resolution.

Aeolun · 2017-04-27T07:36:18Z

Any progress on this? 😢

naderman · 2017-10-18T01:55:03Z

This seems to be caused at least partially by #6753

Bilge · 2019-04-05T21:19:59Z

With such a clear and comprehensive write-up, and a patch to boot, this shouldn't be left hanging so long...

dzuelke · 2019-04-05T23:04:32Z

Well. It's tough to fix since it's a conceptual deficiency. However, @Seldaek promised they'd fix it for Composer v2 ;)

Bilge · 2019-04-05T23:15:52Z

I'll be dead before then.

Slader10 · 2019-09-27T19:20:08Z

is any update for this ? im having this error, just followed every instruction i found but nothing seems to work and the "alcaeus/mongo-php-adapter": "dev-composer-provide", is no longer available

roblem 1
remote: ! > - Installation request for mongodb/mongodb 1.5.0 -> satisfiable by mongodb/mongodb[1.5.0].
remote: ! > - mongodb/mongodb 1.5.0 requires ext-mongodb ^1.6 -> no matching package found.

alcaeus · 2019-09-30T06:23:36Z

Please read the installation instructions for the package in question and you'll see that this has been solved in a better way.

Seldaek · 2020-01-31T16:44:22Z

This should work in 2.0 so closing

Seldaek added the Bug label Mar 9, 2016

Seldaek added this to the Bugs milestone Mar 9, 2016

alcaeus mentioned this issue Mar 10, 2016

Minimum required set to get up and running on Heroku fails: Package is not installed: ext-mongo-1.6.12.0 alcaeus/mongo-php-adapter#67

Closed

Seldaek modified the milestones: 1.1, Bugs Apr 11, 2016

Seldaek modified the milestones: 1.1, 1.2 Apr 28, 2016

stof mentioned this issue May 10, 2016

Add proper provides rules for mbstring and iconv polyfills symfony/polyfill#54

Closed

jmikola referenced this issue in doctrine/doctrine-website-sphinx Jun 1, 2016

Add release blog post for ODM 1.1.0 and 1.0.6

2fec78d

alcaeus mentioned this issue Jul 4, 2016

ext-mongo requirement make problems doctrine/mongodb#269

Closed

mikeddeeson mentioned this issue Oct 17, 2016

Mongo DB php driver requirements teamdeeson/warden#60

Closed

Seldaek modified the milestones: 1.4, 1.3 Nov 6, 2016

alcaeus mentioned this issue Dec 21, 2016

Composer update fails when providing platform package #5987

Closed

jordisala1991 mentioned this issue Feb 25, 2017

Add mongodb to PHP7 with ext-mongo compatibility sonata-project/dev-kit#238

Merged

Seldaek modified the milestones: 1.4, 2.0 Mar 7, 2017

Seldaek mentioned this issue Mar 7, 2017

package provide extension not used in resolve dependencies #6177

Closed

jasverix mentioned this issue Jul 4, 2017

Feature: require one package OR another #6525

Closed

jordisala1991 mentioned this issue Oct 14, 2017

Add composer flags to every update command, remove quiet flag sonata-project/dev-kit#305

Closed

Spomky mentioned this issue May 29, 2018

mbstring extension is not required anymore beberlei/assert#149

Closed

covex-nn mentioned this issue Jun 27, 2018

Provide ext-mongo sonata-project/media-odm-pack#3

Merged

covex-nn mentioned this issue Jul 4, 2018

Add --ignore-platform-reqs option to composer require command symfony/recipes-contrib#409

Merged

stof mentioned this issue Jan 30, 2020

[RFC] Provide unpolyfill meta-package symfony/polyfill#225

Closed

Seldaek closed this as completed Jan 31, 2020

Seldaek modified the milestones: 2.x, 2.0-core Feb 12, 2020

teohhanhui mentioned this issue Mar 8, 2020

Add a stub package for ext-mongodb api-platform/core#3431

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[with patch] requiring a platform package that is provided by another required package fails #5030

[with patch] requiring a platform package that is provided by another required package fails #5030

[with patch] requiring a platform package that is provided by another required package fails #5030

[with patch] requiring a platform package that is provided by another required package fails #5030

Comments