Latest MongoDB images require AVX instructions - Debian VMs fail to run #4321
Comments
|
I tried the example docker-compose and it has image: mongo:4.4 in. I'm still getting this I guess it's just a warning but it's disconcerting that the mongo log has zero other entries apart from this. |
|
I tried to create a development machine on an old Intel Atom board, so I can better check and test the migration from (still broken) snap to docker: but then I have the issue, that I can't test Mongo 5+ as I also get this error message: |
|
I don't know is there any other MongoDB 5+ compatible database. So I presume using WeKan on that old Intel Atom board waits some future time where WeKan could have support for other databases than MongoDB. |
|
see this post from MongoDB folks: https://www.mongodb.com/community/forums/t/mongodb-5-0-cpu-intel-g4650-compatibility/116610 @bryanpedini run This is not an issue of OS support or Wekan support- it's a hard requirement of MongoDB that your CPU has AVX support. |
|
Thanks for info! That's not the only problem with MongoDB:
My MongoDB to SQLite scripts are still in progress, and will also need many code changes. I'm not so sure will there be progress with lungo. Mostly I just like SQL better. Some NewSQL databases are also very interesting with their git branching like features, but I have not tested them well enough yet. Similar could be maybe emulated by saving just IDs and deduplicating database records to save space, but that would complicate queries a lot. |
|
For migrations, I did also take a little look at MongoDB raw database format, and tried to compile it, but that's maybe going too far. I'm not sure yet. Sure it would be useful to convert directly from raw files, that would make it unnecessary to use 4 hours to restore 500 GB mongodump back to running MongoDB server. https://www.percona.com/blog/2021/05/18/wiredtiger-file-forensics-part-1-building-wt/ |
|
I did also try to start FerretDB with docker-compose but did not figure it out well yet, does it work at all: |
|
Yeah I understand you want to migrate away from MongoDB (join the club!), but specifically regarding the OP's issue of MongoDB requiring AVX instructions, there's really nothing he can do except limit installs of MDB to <v5.0 until they upgrade their CPU to something on this list of CPUs supporting AVX. |
|
Does Qemu support emulating AVX ? ;) |
|
Or, should I fork MongoDB, and remove AVX code? ;) |
|
MongoDB is really huge amount of code. I tried compiling it, but it was just too much. SQLite compiles fast. |
Not sure- in my Proxmox environment (which uses Linux KVM) I had to switch my docker host vm from the default "kvm64" vCPU type to the "host" type which essentially just passes thru to the host. So kvm64 doesn't support AVX, but I dont have a list of what all does/not. |
|
@xet7 what are real minimum wekan's mongodb requirements? afair, there's nothing what forces (will force) to upgrade to 5x, right? besides, somebody could care about mongodb's license
tbh, I don't understand a lot of things happening
|
|
Your question moved to #4578 (comment) |
I'm no massive company with giant datacenter, I have a homelab with two different old computers (and admittedly one fairly recent 1U server) running a Proxmox cluster... But it also wouldn't matter actually, since even if the host fail and the vm is spin up from cold boot since everything crashed, you would still encounter these problems where the cpu of the old pcs doesn't support AVX, which is fairly recent as of hardware integration. I still don't understand why we've come along and built decades of software going throug physical capacitors the size of a tube then the microcontroller then punch cards then the floppy drives then cds then dvds then multicore cpus then the 5GHz barrier then now chiplet designs to fit 64c/128t on a single mainstream prosumer threadripper cpu and we now decided we need avx to store data in a fridging database? wtf is wrong with us all humans that we absolutely need to overcomplicate our lives otherwise we're not happy about it? Just document the fact that VMs and the like, without AVX support, don't play nice alongside mongo 5, you need to upgrade to latest mongo 4 but can't jump. yes a little bit of a hot take but hey, we're still talking about it and I don't think mongo will accept any potential wekan's suggestion to drop AVX requirements just because a rando guy on a rando project had a problem with it |
|
btw, I'm also running wekan & mongodb in a kvm-based docker host (also proxmox) and default qemu's
it was Sandy Bridge ~2008, I clearly see mongodb's dev motivation
frankly speaking, it's the first db I've seen to require it be default, normally low-level guys do feature detection at runtime, e.g. ffmpeg upstream ticket: https://www.mongodb.com/community/forums/t/mongodb-5-0-cpu-intel-g4650-compatibility/116610/2 |
I presume most WeKan users are using free Community version of MongoDB, so we are not paying customers of MongoDB. |
Exactly
And exactly this too. Those are precisely the point, IMHO a db doesn't need AVX, and by having such a requirement it defeats the purpose of docker &/or virtualization. So either you compile mongo yourself removing that requirement (might be buggy & you're on your own), or you do a centralized mongo instance on a machine without HA and without containers and all your applications don't have their little mongo instance but all connect to that central one, or you set "host" in your vm and risk losing the ability to migrate over HA scenarios due to cpu discrepancies... Doesn't the mongo team see that this doesn't benefit anyone? What is avx even used for TO STORE DATA on a fridging noSQL db?? Native hardware accelerated encryption or what? IMHO wekan could just move over to anything but mongo (honestly I haven't seen anything a mariadb relational db can't do that a nosql can, but that's just me). |
|
Mongo is huge amount of code to compile. I failed to try to compile it, even on bare metal server with big amount of cores it takes a lot of steps and time, not recommended. SQLite is easy and fast to compile. MariaDB and PostgreSQL do not run on DOS, Amiga and CloudFlare Workers D1. SQLite does run. There is rqlite for client-server version of SQLite, if needed. There is LiteStream fos streaming backups of SQLite. There is BedrockDB for massive scale and additional features. Some have moved from PostgreSQL to SQLite because PostgreSQL is more expensive to manage, problems with VACUUM that takes a long time, etc. SQLite is preferred database for https://sandstorm.io |
Let's just fork the project, move to SQLite, and f*k Mongo and its requirements then... Let's face it, (almost) nobody runs metal anymore, we all use either hypervisors or containers. All tho in the second case since Kube and the pods run on the host itself, there's the argument to be made that the host supports AVX hence not being a vm with |
|
A thread talking about CPU instruction sets and having to compiling your own database. For a Kanban board. 😄 |
Thanks Mongo, I guess 😅 |
|
@timdonovanuk, have you tried to build mongodb without sandy bridge optimizations? |
|
Cool, thanks a lot! I'll try to compile it. |
|
@xet7, |
|
Doh, I got stuck somewhere not getting lzma installed properly. Oh well. |
|
I try to add support for FerretDB where it will be possible to replace MongoDB with FerretDB proxy to SQLite or PostgreSQL #787 (comment) |
|
I can already guess the answer to this is "no, because of how snap works", but would it be possible for Wekan (installed as a snap) to use mongodb tools already installed on the host system? The reason I'm asking is that I'm running Unifi's network application on the same non-AVX-capable host, and the installation script for it has added a mongodb repository patched to work without AVX, so that's already covered. |
|
I did not know about that AVX patched repository, I will try that. Using that patched MongoDB in WeKan Snap would make it possible to run WeKan where is no AVX. I try to fix that WeKan upgrade script, I think attachments are not visible in some cases. There are Snap commands for stopping WeKan included MongoDB, setting custom MongoDB URL to use external MongoDB, but I don't know what would happen with automatic updates and such corner cases. |
|
Right, could be more trouble than it's worth. I have been eyeing some slightly newer (used) hardware for my home server, but for now this AVX requirement from (the Wekan-integrated) MongoDB is the only real reason to upgrade; otherwise the old HP (from 2011!) is still chugging along just fine in what it's used for. :) |
|
There is no reason to upgrade hardware, and add electronic waste. You are not the only one with hardware that does not support AVX. |
|
With this mongo-qemu-avx repo, I did run newest WeKan Docker version on my Core 2 Duo laptop that has 8 GB RAM and CPU does not have AVX. I installed qemu with https://github.com/stevekerrison/mongo-qemu-avx It is possible to check CPU AVX support like this: So, if CPU does not support AVX, it would be possible to automatically run MongoDB 6 at Qemu that supports AVX. That's possible for Docker, Snap, etc. The other way would be to figure out precompiled repo binaries from Unifi, but those are custom compiled and could have modifications by Unifi. It's much easier to run official binaries from MongoDB directly with Qemu, that try to compile MongoDB, MongoDB is huge amount of code, and very many steps to build. |
|
Yes, the qemu solution seems simpler and more future-proof than the custom Unifi repository. Also, my guess would be that few people running Wekan on hardware old enough not to have AVX will have a massively large user base on their installation; most are probably small-time hobbyists like myself, so any negligible slowdowns being exacerbated by scale won't be much of an issue. |
Hi! I noticed you mention my repo. FYI I just updated the above repository to no longer require users to install You can also now specify the mongo tag you want with a build argument. |
|
Thanks a lot ! Your qemu solution is a great way to get MongoDB running on hardware where CPU does not have AVX. |
Issue
After the latest update the
mongodbimage keeps restarting complaining about AVX instructions.Server Setup Information
00478f580306ee13a1eacac9/etc/os-release:uname -a:docker image ls: (before and after today's update)Problem description
After the latest update (today) the
wekandbcontainer is continously restarting with the following error message:Reproduction Steps
docker-compose.ymlin a Debian Buster virtual machine (at least this is my setup)Logs
The
wekanimage is working but cannot connect to the database, thewekandblogs are provided above.Proposed solution
Either:
apt update && apt upgrade -ycan reference something and know what's actually going on before digging through GitHub issues and Jira tickets and all that good stuffThe text was updated successfully, but these errors were encountered: