8000 Avatars and attachments not displaying after upgrading from 6.63 and mongodb 5 to 6.93 and mongodb 6 · Issue #4952 · wekan/wekan · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Avatars and attachments not displaying after upgrading from 6.63 and mongodb 5 to 6.93 and mongodb 6 #4952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
barthelemy-tek opened this issue Jun 8, 2023 · 9 comments
Closed

Avatars and attachments not displaying after upgrading from 6.63 and mongodb 5 to 6.93 and mongodb 6 #4952

barthelemy-tek opened this issue Jun 8, 2023 · 9 comments
Labels
Bug Feature:Cards:Attachments Feature:User-accounts:Avatars

Comments

@barthelemy-tek
Copy link
barthelemy-tek commented Jun 8, 2023 •
edited
Loading

Issue

Server Setup Information

  • Did you test in newest Wekan?: Yes
  • Did you configure root-url correctly so Wekan cards open correctly (see https://github.com/wekan/wekan/wiki/Settings)? Yes
  • Operating System: Host machine is a CentOS 7
  • Deployment Method (Snap/Docker/Sandstorm/bundle/source): Docker (Swarm)
  • Http frontend if any (Caddy, Nginx, Apache, see config examples from Wekan GitHub wiki first): Traefik
  • Node.js Version: The one in Wekan image
  • MongoDB Version: 6
  • What webbrowser version are you using (Wekan should work on all modern browsers that support Javascript)? Firefox/Chrome

Problem description

After upgrading to Wekan 6.93 and MongoDB 6 (From Wekan 6.63 and MongoDB5), all the existing attachments and avatars stopped displaying in Wekan. Newly uploaded images seem to work fine.

When I browse the DB, I can see that avatar links in the user profile are set to, for example, /cfs/files/avatars/qPmz2S7YP4hRZBLR4/téléchargement.png, instead of something like http://localhost/cdn/storage/avatars/YL82r4CYBajvp9dem/original/YL82r4CYBajvp9dem.png?auth=false&brokenIsFine=true with a newly uploaded avatar. I couldn't find the link to an attachment in the cards records, but I suppose something similar is happening there.

Also, the images are still in the cfs_grids.avatars.files and cfs_grids.attachments.files records in the database, as well as in the /data/avatars data/attachments folders in the container.

I tried to run the following queries to try and replay the migration as suggested in another issue, but it did not work.

db.getCollection("meteor-migrations").remove({"name":"migrate-attachments-collectionFS-to-ostrioFiles"})
db.getCollection("meteor-migrations").remove({"name":"migrate-avatars-collectionFS-to-ostrioFiles"})

If we downgrade the MongoDB container to MongoDB 5, the existing images are correctly displayed, but new images aren't.

Is there any way to convert the incorrect links to correct ones ?

Reproduction Steps

Upgrading from Wekan 6.63 to Wekan 6.93 with avatars and attachments I guess?

Logs

wekan-app  | i18next: languageChanged en
wekan-app  | i18next: initialized {
wekan-app  |   debug: 'false',
wekan-app  |   initImmediate: true,
wekan-app  |   ns: [ 'translation' ],
wekan-app  |   defaultNS: 'translation',
wekan-app  |   fallbackLng: [ 'en' ],
wekan-app  |   fallbackNS: false,
wekan-app  |   supportedLngs: [
wekan-app  |     'ar-DZ',  'ar-EG', 'ar',    'ary',   'az-AZ',   'az-LA',
wekan-app  |     'az',     'bg',    'br',    'ca',    'ca-ES',   'cnm',
wekan-app  |     'cs',     'cs-CZ', 'cy-GB', 'cy',    'da',      'de-AT',
wekan-app  |     'de-CH',  'de',    'el-GR', 'el',    'en-BR',   'en-DE',
wekan-app  |     'en-GB',  'en-IT', 'en-MY', 'en-YS', 'en',      'eo',
wekan-app  |     'ast-ES', 'es-AR', 'es-CL', 'es-CO', 'es-LA',   'es-MX',
wekan-app  |     'es-PE',  'es-PY', 'es',    'et-EE', 'eu',      'fa-IR',
wekan-app  |     'fa',     'fi',    'fr-BE', 'fr-CA', 'fr-CH',   'fr',
wekan-app  |     'fy-NL',  'fy',    'gl-ES', 'gl',    'gu-IN',   'he-IL',
wekan-app  |     'he',     'hi-IN', 'hi',    'hr',    'hu',      'hy',
wekan-app  |     'id',     'ig',    'it',    'ja',    'ja-Hira', 'ja-JP',
wekan-app  |     'ka',     'km',    'ko-KR', 'ko',    'lt',      'lv',
wekan-app  |     'mk',     'mn',    'ms',    'ms-MY', 'nb',      'nl-NL',
wekan-app  |     'nl',     'oc',    'pa',    'pl-PL', 'pl',      'pt-BR',
wekan-app  |     'pt',     'pt-PT', 'ro',    'ro-RO', 'ru-UA',   'ru',
wekan-app  |     'sk',     'sl',    'sr',    'sv',    'sw',      'ta',
wekan-app  |     'th',     'tlh',   'tr',    'ua-UA',
wekan-app  |     ... 30 more items
wekan-app  |   ],
wekan-app  |   nonExplicitSupportedLngs: false,
wekan-app  |   load: 'all',
wekan-app  |   preload: false,
wekan-app  |   simplifyPluralSuffix: true,
wekan-app  |   keySeparator: '.',
wekan-app  |   nsSeparator: ':',
wekan-app  |   pluralSeparator: '_',
wekan-app  |   contextSeparator: '_',
wekan-app  |   partialBundledLanguages: false,
wekan-app  |   saveMissing: false,
wekan-app  |   updateMissing: false,
wekan-app  |   saveMissingTo: 'fallback',
wekan-app  |   saveMissingPlurals: true,
wekan-app  |   missingKeyHandler: false,
wekan-app  |   missingInterpolationHandler: false,
wekan-app  |   postProcess: [ 'sprintf' ],
wekan-app  |   postProcessPassResolved: false,
wekan-app  |   returnNull: true,
wekan-app  |   returnEmptyString: true,
wekan-app  |   returnObjects: false,
wekan-app  |   joinArrays: false,
wekan-app  |   returnedObjectHandler: false,
wekan-app  |   parseMissingKeyHandler: false,
wekan-app  |   appendNamespaceToMissingKey: false,
wekan-app  |   appendNamespaceToCIMode: false,
wekan-app  |   overloadTranslationOptionHandler: [Function: handle],
wekan-app  |   interpolation: {
wekan-app  |     escapeValue: false,
wekan-app  |     format: [Function: bound format],
wekan-app  |     prefix: '__',
wekan-app  |     suffix: '__',
wekan-app  |     formatSeparator: ',',
wekan-app  |     unescapePrefix: '-',
wekan-app  |     nestingPrefix: '$t(',
wekan-app  |     nestingSuffix: ')',
wekan-app  |     nestingOptionsSeparator: ',',
wekan-app  |     maxReplaces: 1000,
wekan-app  |     skipOnVariables: true
wekan-app  |   },
wekan-app  |   cleanCode: true,
wekan-app  |   defaultNs: 'translation',
wekan-app  |   resources: {},
wekan-app  |   ignoreJSONStructure: true,
wekan-app  |   lng: 'en'
wekan-app  | }
wekan-app  | {"line":"87","file":"percolate_synced-cron.js","message":"SyncedCron: Scheduled \"notification_cleanup\" next run @Thu Jun 08 2023 06:45:44 GMT+0000 (Coordinated Universal Time)","time":{"$date":1686206744738},"level":"info"}
wekan-app  | (node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
wekan-app  | (Use `node --trace-deprecation ...` to show where the warning was created)
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'gridFsFileId' of undefined
wekan-app  |     at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:42:83)
wekan-app  |     at FilesCollection.interceptDownload (models/avatars.js:109:42)
wekan-app  |     at FilesCollection.download (packages/ostrio:files/server.js:1780:88)
wekan-app  |     at packages/ostrio:files/server.js:747:22
wekan-app  |     at /build/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 8)
wekan-app  | (node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'gridFsFileId' of undefined
wekan-app  |     at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:42:83)
wekan-app  |     at FilesCollection.interceptDownload (models/avatars.js:109:42)
wekan-app  |     at FilesCollection.download
8000
 (packages/ostrio:files/server.js:1780:88)
wekan-app  |     at packages/ostrio:files/server.js:747:22
wekan-app  |     at /build/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 9)
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'gridFsFileId' of undefined
wekan-app  |     at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:42:83)
wekan-app  |     at FilesCollection.interceptDownload (models/avatars.js:109:42)
wekan-app  |     at FilesCollection.download (packages/ostrio:files/server.js:1780:88)
wekan-app  |     at packages/ostrio:files/server.js:747:22
wekan-app  |     at /build/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 10)
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'gridFsFileId' of undefined
wekan-app  |     at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:42:83)
wekan-app  |     at FilesCollection.interceptDownload (models/avatars.js:109:42)
wekan-app  |     at FilesCollection.download (packages/ostrio:files/server.js:1780:88)
wekan-app  |     at packages/ostrio:files/server.js:747:22
wekan-app  |     at /build/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 11)
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'gridFsFileId' of undefined
wekan-app  |     at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:42:83)
wekan-app  |     at FilesCollection.interceptDownload (models/avatars.js:109:42)
wekan-app  |     at FilesCollection.download (packages/ostrio:files/server.js:1780:88)
wekan-app  |     at packages/ostrio:files/server.js:747:22
wekan-app  |     at /build/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 12)
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'gridFsFileId' of undefined
wekan-app  |     at FileStoreStrategyFactory.getFileStrategy (models/lib/fileStoreStrategy.js:42:83)
wekan-app  |     at FilesCollection.interceptDownload (models/avatars.js:109:42)
wekan-app  |     at FilesCollection.download (packages/ostrio:files/server.js:1780:88)
wekan-app  |     at packages/ostrio:files/server.js:747:22
wekan-app  |     at /build/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/fiber_pool.js:43:40
wekan-app  | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 13)
@xet7
Copy link
Member
xet7 commented Jun 8, 2023 •
edited
Loading

@barthelemy-tek

When migrating, there needs to be setting WRITABLE_PATH to migrate attachments properly:
https://github.com/wekan/wekan/blob/master/docker-compose.yml#L156

And if there are any file there, they need to be there also with newest MongoDB and WeKan.

Migration process takes some time in background, before images appear. Images are not visible immediately.

I try to figure out is there some way to improve this migration. It could be nice if attahments would not be temporarily stored to disk and read back, but instead there would be in-place changes only to file text details, not to actual binary files. Any help very welcome.

@xet7
Copy link
Member
xet7 commented Jun 8, 2023

@barthelemy-tek

Attachment migrations are done here, this code needs improvements:
https://github.com/wekan/wekan/blob/master/server/migrations.js#L1248-L1413

@mfilser
Copy link
Contributor
mfilser commented Jun 8, 2023

Between 6.63 and 6.93 there were no attachment migrations.
I think you manually removed the migration tags from database so the migration started again, that could cause some to many problems because the migration code isnt working for this situation.
About the url at database, there is no absolute link including the domain, only a relativ url, otherwise it would not be possible to change the domain of the wekan instance.
I guess the update to MongoDB 6 wasn't right / successfully done. Use Mongo 5 if it's working for you or have a look to the update steps again. In general, MongoDB 6 is working with latest Wekan.

@barthelemy-tek
Copy link
Author

Thanks for your reply.

We had set the WRITABLE_PATH variable in our docker compose, so I don't think the problem lies there. Downgrading to Mongo 5 is not a solution, as new avatars and images are not displayed correctly.

We have spent too much time on this problem already, I think we are going to bear with that and tell our users to re-upload their avatars, and to contact us if they need any attachment they uploaded previously.

@barthelemy-tek barthelemy-tek changed the title Avatars and attachments not displaying after upgrading from 6.63 and mpngodb 5 to 6.93 and mongodb 6 Avatars and attachments not displaying after upgrading from 6.63 and mongodb 5 to 6.93 and mongodb 6 Jun 8, 2023
@xet7
Copy link
Member
xet7 commented Jul 20, 2023

Related #4780 (comment)

@akitzing
Copy link

we have the same problem

@xet7
Copy link
8000 Member
xet7 commented Jul 20, 2023

@akitzing

Yes, I added steps to #4780 (comment) how anyone can help with fixing this.

@Jaffex
Copy link
Jaffex commented Sep 12, 2023 •
edited
Loading

I'm having a similar problem (though I'm not sure if it is exactly the same since for me, previews also don't get shown after the update) and avatars do get shown.

Here's how I migrated from version 5.65 with mongodb version 4.2 to version 7.09 with mongodb version 6:

# Backing up the database
docker stop wekan-app
docker exec wekan-db mongodump
docker cp wekan-db:dump /some/nice/place
# Stopping the service
docker-compose stop
# Removing the databases persistent file storage
docker volume rm wekan-db
# Replacing the docker-compose.yml with a prepared file with everything needed for version 7.09
cp docker-compose-7.09.yml docker-compose.yml
# Starting the database and restoring the dump made earlier 
docker-compose up wekandb -d
docker cp /some/nice/place wekan-db:/dump
docker exec wekan-db mongorestore --drop --db wekan /dump
# Starting wekan itself
docker-compose up -d

When I look into the wekan-app container after this, it seems part of the migration worked, since the /data/attachments directory now contains a lot of files.

If I however look to a board with a card that had an attachment before the upgrade, it does not show it anymore:

grafik
Card before the update shows 2 attachments

grafik
Same card after the update shows no attachments

@xet7
Copy link
Member
xet7 commented Nov 21, 2023

There will be mongodb migration tools available later.

@xet7 xet7 closed this as completed Nov 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug Feature:Cards:Attachments Feature:User-accounts:Avatars
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@xet7 @mfilser @Jaffex @akitzing @barthelemy-tek
0