8000 yarn.lock to package-lock.json conversion ignores workspaces · Issue #44 · imsnif/synp · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yarn.lock to package-lock.json conversion ignores workspaces #44

Closed
qwelias opened this issue Aug 14, 2020 · 9 comments
Closed

yarn.lock to package-lock.json conversion ignores workspaces #44

qwelias opened this issue Aug 14, 2020 · 9 comments

Comments

@qwelias
Copy link
qwelias commented Aug 14, 2020

Issue description:

Workspaces may have specific versions of dependencies which would not get hoisted to the root level, but synp accounts only for root node_modules Looks like there's an issue with nmtree, which leads to a loss of information in resulting package-lock.json

test repo

produced package-lock.json does not have other lodash versions:

{"name":"test-yarn-audit-fix","version":"1.0.0","lockfileVersion":1,"requires":true,"dependencies":{"lodash":{"version":"3.10.1","resolved":"https://registry.yarnpkg.com/lodash/-/lodash-3.10.1.tgz","integrity":"sha1-W/Rejkm6QYnhfUgnid/RW9FAt7Y="}}}

UPD: fixed repo link

@antongolub
Copy link
Collaborator
antongolub commented Aug 14, 2020

@qwelias
Take a look: https://github.com/imsnif/nmtree/blob/master/index.js#L25

if (fs.existsSync(nodeModulesPath)) { // TODO: and is dir
      const nodeModulesPackages = fs.readdirSync(nodeModulesPath)
      nodeModulesPackages
      .filter(dir => !/^\./.test(dir)) // no hidden directories (eg. .bin)
      .reduce((packages, dir) => mergeScopes(nodeModulesPath, packages, dir), [])
      .forEach(nmPackage => nmTree(path.join(nodeModulesPath, nmPackage), tree))
    }

As I remember, fs.readdirSync ignores symlinks by default. But we could try running dir scanner with withFileTypes option.

UPD: It reads.

It just overrides all the same named deps with the last found.

@antongolub
Copy link
Collaborator

I'll try to fix it on this weekend.

@imsnif
Copy link
Owner
imsnif commented Aug 19, 2020

Hey, @qwelias - I think now with @antongolub's fixed published we can close this, right?

@qwelias
Copy link
Author
qwelias commented Aug 20, 2020

Hi, there's something strange. @antongolub

If you run synp --source-file yarn.lock in the test repo, you'd get Cannot read property 'pkgPath' of undefined error.
I tried to track it and it seems like:

  • it uses lib/lockfileV2/index.js
  • getWorkspaces return empty array (looks like it does not understand the wildcard)
  • the error is thrown at lib/lockfileV1/tree.js#56 because empty array of workspaces produced empty object

@antongolub
Copy link
Collaborator

@qwelias,
There is a small fix: #47
Patch version hasn't been released yet, so I published it under my account: @antongolub/synp@2.0.12 for testing.
Could you check this out?

@qwelias
Copy link
Author
qwelias commented Aug 20, 2020

I think it still doesn't work

output on test repo
qwelias@schibmaze ~/prj/stuff/test-yarn-audit-fix master ● (1.0.0)                                                                                                        ⬡ 12.18.1 [13:15:10]
$ synp --source-file yarn.lock && cat yarn.lock package-lock.json 
Created package-lock.json
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1


lodash@4.0.0:
  version "4.0.0"
  resolved "https://registry.yarnpkg.com/lodash/-/lodash-4.0.0.tgz#9ac43844c595e28d30108b7ba583703395922dfc"
  integrity sha1-msQ4RMWV4o0wEIt7pYNwM5WSLfw=

lodash@4.17.0:
  version "4.17.0"
  resolved "https://registry.yarnpkg.com/lodash/-/lodash-4.17.0.tgz#93f4466e5ab73e5a1f1216c34eea11535f0a8df5"
  integrity sha1-k/RGblq3PlofEhbDTuoRU18KjfU=

lodash@^3.0.0:
  version "3.10.1"
  resolved "https://registry.yarnpkg.com/lodash/-/lodash-3.10.1.tgz#5bf45e8e49ba4189e17d482789dfd15bd140b7b6"
  integrity sha1-W/Rejkm6QYnhfUgnid/RW9FAt7Y=
{
  "name": "test-yarn-audit-fix",
  "version": "1.0.0",
  "lockfileVersion": 2,
  "requires": true,
  "packages": {
    "": {
      "name": "test-yarn-audit-fix",
      "version": "1.0.0",
      "license": "MIT",
      "workspaces": [
        "packages/*"
      ],
      "dependencies": {
        "lodash": "^3.0.0"
      }
    },
    "node_modules/lodash": {
      "version": "3.10.1",
      "resolved": "https://registry.yarnpkg.com/lodash/-/lodash-3.10.1.tgz",
      "integrity": "sha1-W/Rejkm6QYnhfUgnid/RW9FAt7Y=",
      "license": "MIT"
    }
  },
  "dependencies": {
    "lodash": {
      "version": "3.10.1",
      "resolved": "https://registry.yarnpkg.com/lodash/-/lodash-3.10.1.tgz",
      "integrity": "sha1-W/Rejkm6QYnhfUgnid/RW9FAt7Y="
    }
  }
}%            

@antongolub
Copy link
Collaborator

Thanks for the feedback. I'll let you know when I fix this issue.

@qwelias
Copy link
Author
qwelias commented Aug 20, 2020

Btw test repo was linked in the initial issue comment, so you can test the implementation :)
I'd like to provide more help, but am occupied with other things until the weekend

antongolub added a commit that referenced this issue Aug 20, 2020
imsnif pushed a commit that referenced this issue Aug 21, 2020
* refactor: separate workspace utils

* fix: adapt parentPackagePath resolver to relative paths

closes #46

* fix(lockfileV2): add some missed relative paths handlers, add test for workspaces with cross-refs

relates #44

* feat: enable lockfile v2 (workspaces) processing by cli flag

* fix: use platform specific separators

* fix: adapt parent path resolver to windows-based runtime

* feat: replace workspace exception flow with warning
antongolub added a commit that referenced this issue Aug 31, 2020
# 1.0.0 (2020-08-31)

### Bug Fixes

* better cli errors ([1e05ade](1e05ade))
* check node_modules properly ([d866ee2](d866ee2))
* corrupted entries in yarn.lock file ([a8ef713](a8ef713))
* dependencies do not have duplicates and add optional deps ([a3289bc](a3289bc))
* eol formatting ([fe50c68](fe50c68))
* git & tarball support yarn -> npm ([#28](#28)) ([c0703ef](c0703ef))
* move tap-spec to devDeps ([4b57858](4b57858))
* parent path bug with relative paths ([4b8ed4a](4b8ed4a))
* parentPackagePath resolver ([#47](#47)) ([6395a7e](6395a7e)), closes [#46](#46) [#44](#44)
* remove redundant exit condition ([73320c0](73320c0))
* support crlf line endings ([f555d0e](f555d0e))
* support npm integrities with several hashes, use legacy url.parse if exists ([610bf77](610bf77))
* **npm-to-yarn:** support github deps ([43a485d](43a485d))
* **tree:** properly handle root level hoisting conversion ([9a7eec9](9a7eec9))

### Features

* **ecosystem:** handle yarn workspaces and integrity fields ([#45](#45)) ([c84fb2c](c84fb2c))
* add primary TS typings ([#42](#42)) ([512378e](512378e))
* **cli:** --force ([#37](#37)) ([57a081c](57a081c))
* **conversion:** improve file dependencies support ([c1883f8](c1883f8))
* bundled dependencies and root devDependencies ([e26ae6c](e26ae6c))
* cli usage ([2b3b711](2b3b711))
* multiple level dependencies ([4ec40cc](4ec40cc))
* use node_modules dir for state ([31a1e6d](31a1e6d))
antongolub added a commit that referenced this issue Aug 31, 2020
# 1.0.0 (2020-08-31)

### Bug Fixes

* better cli errors ([1e05ade](1e05ade))
* check node_modules properly ([d866ee2](d866ee2))
* corrupted entries in yarn.lock file ([a8ef713](a8ef713))
* dependencies do not have duplicates and add optional deps ([a3289bc](a3289bc))
* eol formatting ([fe50c68](fe50c68))
* git & tarball support yarn -> npm ([#28](#28)) ([c0703ef](c0703ef))
* move tap-spec to devDeps ([4b57858](4b57858))
* parent path bug with relative paths ([4b8ed4a](4b8ed4a))
* parentPackagePath resolver ([#47](#47)) ([6395a7e](6395a7e)), closes [#46](#46) [#44](#44)
* remove redundant exit condition ([73320c0](73320c0))
* support crlf line endings ([f555d0e](f555d0e))
* support npm integrities with several hashes, use legacy url.parse if exists ([610bf77](610bf77))
* **npm-to-yarn:** support github deps ([43a485d](43a485d))
* **tree:** properly handle root level hoisting conversion ([9a7eec9](9a7eec9))

### Features

* **ecosystem:** handle yarn workspaces and integrity fields ([#45](#45)) ([c84fb2c](c84fb2c))
* add primary TS typings ([#42](#42)) ([512378e](512378e))
* **cli:** --force ([#37](#37)) ([57a081c](57a081c))
* **conversion:** improve file dependencies support ([c1883f8](c1883f8))
* bundled dependencies and root devDependencies ([e26ae6c](e26ae6c))
* cli usage ([2b3b711](2b3b711))
* multiple level dependencies ([4ec40cc](4ec40cc))
* use node_modules dir for state ([31a1e6d](31a1e6d))
@antongolub
Copy link
Collaborator

@qwelias

From v1.8.1 synp provides experimental support for workspaces. This feature could be enabled with --with-workspaces flag.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
0