8000 Fix errors compiling for OSX catalina by eugene-so · Pull Request #940 · Veil-Project/veil · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fix errors compiling for OSX catalina #940

New issue < 8000 /summary>

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 25, 2021

Conversation

eugene-so
Copy link

Push Summary
Fix compilation issue on MacOS 10.15 Catalina machine.

Root Cause
Changes in XCode 11 Objective-C Runtime caused an error building on
MacOS 10.15 Catalina machine.
Updated instruction on how to build berkeley DB for macOS 10.15 Catalina.

Solution
Ported solution from bitcoin PR 16720.

Bounty PR
Bounty Payment Address
sv1qqp6aptgvgp9t9h8sgzkqmu6cgq2e20l9x6fsl5ask7t3ygy2jagftcpq0x5z0h522ca5h06qq3hx33pke00r7gjt3j24n896gf55y68ptrmjqqqqd8lz3

Unit Testing Results

Copy link
Collaborator
@CaveSpectre11 CaveSpectre11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK 1f03f54

@CaveSpectre11 CaveSpectre11 added the Tag: Waiting For Code Review Waiting for code review from a core developer label May 4, 2021
@seanPhill
Copy link
Collaborator
seanPhill commented May 5, 2021

bootstrap.log
Seem to have a problem with Boost on a fairly fresh Catalina MacBook Pro.

cd depends
make

...
Fetching boost_1_64_0.tar.bz2 from https://dl.bintray.com/boostorg/release/1.64.0/source/
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- 0:00:02 --:--:-- 0
curl: (22) The requested URL returned error: 403 Forbidden
Fetching boost_1_64_0.tar.bz2 from https://bitcoincore.org/depends-sources
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 76.7M 100 76.7M 0 0 157k 0 0:08:19 0:08:19 --:--:-- 378k
/Users/.../Documents/GitHub/veil/depends/work/download/boost-1_64_0/boost_1_64_0.tar.bz2.temp: OK
Extracting boost...
/Users/.../Documents/GitHub/veil/depends/sources/boost_1_64_0.tar.bz2: OK
Preprocessing boost...
Configuring boost...
Building Boost.Build engine with toolset darwin...
Failed to build Boost.Build build engine
Consult 'bootstrap.log' for more details
make: *** [/Users/.../Documents/GitHub/veil/depends/work/build/x86_64-apple-darwin19.6.0/boost/1_64_0-0bee59b2cae/./.stamp_configured] Error 1

@eugene-so
Copy link
Author
eugene-so commented May 6, 2021 via email

@seanPhill
Copy link
Collaborator

I got a checksum error on the first download, so it downloaded from the bitcoin repo and got a checksum error also.

@eugene-so
Copy link
Author
eugene-so commented May 6, 2021

Let's talk about this for a minute. It now looks like Cave Spectre's change broke the automated build check (which was working previously). In this PR, I never touched the dependent boost library or the automated build macOS version. It was meant to fix a local build on MacOS Catalina which is what I'm using.

I usually build using these commands which are described in build-osx.md:

brew install automake berkeley-db4 libtool boost miniupnpc openssl pkg-config protobuf python qt libevent qrencode gmp
CFLAGS="-Wno-error=implicit-function-declaration"  ./contrib/install_db4.sh .
export BDB_PREFIX=‘$(pwd)/db4'
./autogen.sh

./configure BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include"
make -j4

As such I never tested
make depends
on my local machine. Why do you need to compile it like this? Are you able to compile it using the same commands as me?

@CaveSpectre11 CaveSpectre11 force-pushed the feature/osx-catalina branch from bd2d1ac to 1f03f54 Compare May 6, 2021 10:27
@CaveSpectre11
Copy link
Collaborator

Let's talk about this for a minute. It now looks like Cave Spectre's change broke the automated build check (which was working previously).

Since you didn't build depends; we don't know if bintray.com going down is what broke depends, or if depends doesn't build on Catalina. So this was an attempt to isolate the problem. Since Sean is able to test it, this seemed like a good place to put that change quickly so that he doesn't have to monkey around with cherry-picking together two different branches.

It's been reverted since boost.org had a bad checksum on their website.

@eugene-so
Copy link
Author

It's been reverted since boost.org had a bad checksum on their website.

What shall we do here then? I know that later in the C++17 branch I do start updating the boost library and automated build osx versions.

@CaveSpectre11 CaveSpectre11 force-pushed the feature/osx-catalina branch from bd2d1ac to 1f03f54 Compare May 7, 2021 00:54
@CaveSpectre11
Copy link
Collaborator

It's been reverted since boost.org had a bad checksum on their website.

What shall we do here then? I know that later in the C++17 branch I do start updating the boost library and automated build osx versions.

We're going to have @seanPhill build this with your method; make sure nothing seems broken, and if that's the case merge it, and cross the depends bridge with your c++17 branch since it doesn't look like there's an easy cherry-pick into this branch to solve that issue.

@seanPhill
Copy link
Collaborator

On Catalina, using the documented in build-osx.md brew install ... method depends installed fine. Carefully checked afterward. (Also successful on Big Sur and High Sierra.)

When compiling afterwards, with

 ./autogen.sh
 ./configure
 make

Everything seemed to be going fine when suddenly getting 20 errors apparently relating to C++17, hitting the maximum errors and then stopping.

I've copied and pasted the output from around the point that the errors began.

Catalina-veil-install-errors-20210507.pdf

@eugene-so
Copy link
Author

On Catalina, using the documented in build-osx.md brew install ... method depends installed fine. Carefully checked afterward. (Also successful on Big Sur and High Sierra.)

When compiling afterwards, with

 ./autogen.sh
 ./configure
 make

Everything seemed to be going fine when suddenly getting 20 errors apparently relating to C++17, hitting the maximum errors and then stopping.

I've copied and pasted the output from around the point that the errors began.

Catalina-veil-install-errors-20210507.pdf

Hmmm... peculiar. I thought I had it compiling on my Catalina system without requiring C++17 at this point. I'll give it another try from scratch build as well.

@eugene-so
Copy link
Author
eugene-so commented May 7, 2021

Ok, I found that this was actually missing one cherry-pick related to dark mode that is required. It now compiles fine on my Catalina system from scratch. @seanPhill : I wonder if there is a mismatch on your system between the QT@5 installed using brew as a dependency and another version of QT on your system? On my system, I do not have the path /usr/local/include/QtCore where your error is showing up. Instead I have /usr/local/Cellar/qt@5/5.15.2/include/QtCore which is the one installed by brew.

It's possible that the version you have on your system already requires C++17 for QT which would require pulling a lot more of the changes from the C++17 branch, whereas the one installed by brew still works fine using C++11.

Here is exactly what I ran to get it to compile and check certain things:

./autogen
./configure BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include"
make -j4

@eugene-so
Copy link
Author

brew info qt@5
qt@5: stable 5.15.2 (bottled) [keg-only]
Cross-platform application and UI framework
https://www.qt.io/
/usr/local/Cellar/qt@5/5.15.2 (10,688 files, 367.9MB)
Poured from bottle on 2021-04-11 at 12:07:56
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/qt@5.rb
License: GFDL-1.3-only and GPL-2.0-only and GPL-3.0-only and LGPL-2.1-only and LGPL-3.0-only
==> Dependencies
Build: pkg-config ✔
==> Requirements
Build: Xcode ✘
Required: macOS >= 10.12 ✔
==> Caveats
We agreed to the Qt open source license for you.
If this is unacceptable you should uninstall.

qt@5 is keg-only, which means it was not symlinked into /usr/local,
because this is an alternate version of another formula.

If you need to have qt@5 first in your PATH, run:
echo 'export PATH="/usr/local/opt/qt@5/bin:$PATH"' >> ~/.profile

For compilers to find qt@5 you may need to set:
export LDFLAGS="-L/usr/local/opt/qt@5/lib"
export CPPFLAGS="-I/usr/local/opt/qt@5/include"

For pkg-config to find qt@5 you may need to set:
export PKG_CONFIG_PATH="/usr/local/opt/qt@5/lib/pkgconfig"

==> Analytics
install: 52,393 (30 days), 142,535 (90 days), 142,565 (365 days)
install-on-request: 4,985 (30 days), 11,022 (90 days), 11,052 (365 days)
build-error: 0 (30 days)

@eugene-so
Copy link
Author

On another note, I wonder why it is pulling the QT5 include from the wrong place?

@seanPhill
Copy link
Collaborator
seanPhill commented May 7, 2021

I have the same qt@5 result that you have. I don't know why an error would refer to a different file path.

 brew info qt@5
qt@5: stable 5.15.2 (bottled) [keg-only]
Cross-platform application and UI framework
https://www.qt.io/
/usr/local/Cellar/qt@5/5.15.2 (10,688 files, 367.9MB)
  Poured from bottle on 2021-04-09 at 11:35:54
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/qt@5.rb
License: GFDL-1.3-only and GPL-2.0-only and GPL-3.0-only and LGPL-2.1-only and LGPL-3.0- Dependencies
Build: pkg-config ✔
==> Requirements
Build: Xcode ✔
Required: macOS >= 10.12 ✔

(Caveats & analytics are identical.)

I don't remember installing any other qt (unless I was trying to install dependencies for another wallet), but brew info qt reveals:

 brew info qt  
qt: stable 6.0.3 (bottled), HEAD
Cross-platform application and UI framework
https://www.qt.io/
/usr/local/Cellar/qt/6.0.3_2 (8,233 files, 158.7MB) *
  Poured from bottle on 2021-05-06 at 21:46:19
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/qt.rb
License: GFDL-1.3-only and GPL-2.0-only and GPL-3.0-only and LGPL-2.1-only and LGPL-3.0- Dependencies
Build: cmake ✘, ninja ✘, pkg-config ✔
Required: assimp ✔, dbus ✔, double-conversion ✔, freetype ✔, glib ✔, icu4c ✔, jasper ✔, jpeg ✔, libb2 ✔, libpng ✔, libproxy ✔, libtiff ✔, pcre2 ✔, python@3.9 ✔, sqlite ✔, webp ✔, zstd ✔
==> Requirements
Build: Xcode ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 59,264 (30 days), 184,791 (90 days), 793,073 (365 days)
install-on-request: 21,701 (30 days), 57,318 (90 days), 194,687 (365 days)
build-error: 0 (30 days)

... Oh! Looking at that date and time that was last night and would have been right when I was installing dependencies for this Veil PR. ... I notice that the brew install ... list listed "qt", not qt@5, and would have installed qt 6, while the earlier date on qt@5 is actually the date I was installing dependencies for a different coin.

@eugene-so
Copy link
Author
eugene-so commented May 7, 2021 via email

@seanPhill
Copy link
Collaborator

:/
I just tried

./autogen.sh
./configure BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" BDB_CFLAGS="-I${BDB_PREFIX}/include"
make -j4

on my Big Sur Mac, bearing in mind that it got just the same errors as the two Catalinas, and I still got the

fatal error: too many errors emitted, stopping now [-ferror-limit=]
11 warnings and 20 errors generated.
make[2]: *** [qt/test/test_veil_qt-addressbooktests.o] Error 1
make[2]: *** Waiting for unfinished jobs....
11 warnings and 20 errors generated.
make[2]: *** [qt/test/test_veil_qt-paymentservertests.o] Error 1
11 warnings and 20 errors generated.

and so on.

Note that Qt5 is apparently getting included from the Cellar and then used in an "include" directory.

In file included from /usr/local/Cellar/qt@5/5.15.2/lib/QtCore.framework/Headers/QByteArray:1:
In file included from /usr/local/Cellar/qt@5/5.15.2/lib/QtCore.framework/Headers/qbytearray.h:44:
In file included from /usr/local/include/QtCore/qrefcount.h:43:
In file included from /usr/local/include/QtCore/qatomic.h:41:
/usr/local/include/QtCore/qglobal.h:667:87: error: expected '>'
         typename = std::enable_if_t<std::is_arithmetic_v<T> && std::is_arithmetic_v<U> &&```

@eugene-so
Copy link
Author
eugene-so commented May 8, 2021 via email

@seanPhill
Copy link
Collaborator
seanPhill commented May 8, 2021

I really don't think that is the issue. /usr/local/include/QtCore is a symlink to QtCore -> ../Cellar/qt/6.0.3_2/include/QtCore
There's are dozens of Qt-related symlinks in there, including QtCore5Compat -> ../Cellar/qt/6.0.3_2/include/QtCore5Compat

In fact it was the brew install ... qt ... that installed this version. A few weeks earlier on a different coin qt@5 was installed for those depends. It's not entirely in scope for this PR, but I have a High Sierra Mac that had neither installed and I h 8000 ave installed qt@5 on it. (I think qt@6 might not be High Sierra compatible.) I will try the installation on that one for the moment, although being out of support there might be other issues.

Please bear in mind the note on the qt@5 installation that

qt@5 is keg-only, which means it was not symlinked into /usr/local, because this is an alternate version of another formula.
If you need to have qt@5 first in your PATH, run:
echo 'export PATH="/usr/local/opt/qt@5/bin:$PATH"' >> ~/.profile

For compilers to find qt@5 you may need to set:
export LDFLAGS="-L/usr/local/opt/qt@5/lib"
export CPPFLAGS="-I/usr/local/opt/qt@5/include"

For pkg-config to find qt@5 you may need to set:
export PKG_CONFIG_PATH="/usr/local/opt/qt@5/lib/pkgconfig"

@eugene-so
Copy link
Author
eugene-so commented May 8, 2021 via email

@seanPhill
Copy link
Collaborator

No worries. FYI, the High Sierra Mac that I specifically made sure that brew install qt@5 instead of brew install qt did correctly compile, without problems.

@seanPhill
Copy link
Collaborator

On a Catalina Mac I removed qt@6 and pyqt that depends on it, and tried to compile, but I got an error relating to files written with an older version of protoc. brew info protobuf shows 'stable 3.15.8'. I'm guessing that some other dependencies other than qt@5 should be specified (in brew install ...) as not the latest.
protobuffer-error.pdf

@eugene-so
Copy link
Author

On a Catalina Mac I removed qt@6 and pyqt that depends on it, and tried to compile, but I got an error relating to files written with an older version of protoc. brew info protobuf shows 'stable 3.15.8'. I'm guessing that some other dependencies other than qt@5 should be specified (in brew install ...) as not the latest.
protobuffer-error.pdf

The brew version 3.15.8 seems correct. What do you get when you type this command?

protoc --version

I get libprotoc 3.15.8

@eugene-so
Copy link
Author

Seems like bitcoin eventually completely dropped protobuf from their build

@eugene-so
Copy link
Author

What is happening with this PR? Seems like something broke with the automated build recently?

@seanPhill
Copy link
Collaborator
seanPhill commented May 21, 2021

I can approve the automated build again now. I see it is waiting.
If that works I will have another try on my Catalina and Big Sur Macs.

Copy link
Collaborator
@codeofalltrades codeofalltrades left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the CPPFLAGS changes need to appear in the yml files used by gitian
https://github.com/Veil-Project/veil/blob/master/contrib/gitian-descriptors/gitian-linux.yml

@eugene-so
Copy link
Author
eugene-so commented Jun 2, 2021

I think the CPPFLAGS changes need to appear in the yml files used by gitian
https://github.com/Veil-Project/veil/blob/master/contrib/gitian-descriptors/gitian-linux.yml

This change should not impact the linux build.
I reviewed gitian-osx.yml and there are no CPPFLAGS in that one. Do you know how I can test the gitian build process to make sure it is working?

@codeofalltrades ^^^

Copy link
Collaborator
@codeofalltrades codeofalltrades left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK d596243

@codeofalltrades codeofalltrades added Code Review: Passed and removed Tag: Waiting For Code Review Waiting for code review from a core developer labels Jun 25, 2021
Changes in XCode 11 Objective-C Runtime caused a build error.
(Solution ported from bitcoin PR 16720)

Comment out incompatible legacy dark mode fix.

Updated instruction on how to build berkeley DB for Catalina.

Specify qt@5 requirement in osx build instructions
Installation of higher versions of qt causes build to fail.

Update  boost depends path to jfrog
@eugene-so eugene-so force-pushed the feature/osx-catalina branch from d596243 to f0120a0 Compare June 25, 2021 06:16
@codeofalltrades codeofalltrades merged commit 5031c6c into Veil-Project:master Jun 25, 2021
@CaveSpectre11 CaveSpectre11 added Dev Status: Merged Issue is completely finished. Bounty: Paid This PR has been paid a bounty labels Jun 27, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bounty: Paid This PR has been paid a bounty Code Review: Passed Dev Status: Merged Issue is completely finished. Tag: Build System Tag: OS - OSX Problems specific to Mac OS
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants
0