Tags: L-Acoustics/fastbuild
Tags
Sync dev to master branch (fastbuild#812) * Travis CI: Update FASTBuild to v1.01 * Extending #if parsing with boolean && and || (fastbuild#733) * Extending #if parsing with boolean && and || * Fix build on gcc/g++ * Updated documentation * Extending #if parsing with boolean && and || * Fix build on gcc/g++ * Updated documentation * Apply && and || operator precedence (&& before ||) * Tokenizer now recognises two character operators && and ||, #if handler modified accordingly * Remove tests from if_directive.bff, add them into TestBFFParsing.cpp * Actually evaluate the first defined result during the || pass * Kludge tests a bit until better understood * Formatting and cleanup * Added errors for #if operator parsing, applied them to tests. Co-authored-by: George Nakos <george.nakos@u-blox.com> Co-authored-by: ffulin <franta.fulin@gmail.com> * Fix bad merge * Delete stale file [skip ci] * Post intergation: #if supports || and && - simplify logic - add tests for some additional cases - simplify tests - code style - add documentation for new errors - update documentation to reflect precedence rules * [Improvement] Increase thread stack size to avoid unexpected problems with cache plugins * CachePlugin improvements [Note] DB Version changed [Improvement] Cache plugin can be initialized via InitEx which provides details of operating mode (read/write/verbose) [Improvement] Settings exposes .CachePluginDLLConfig to all passing of custom config to cache plugins [Improvement][Windows] Cache plugin DLL function names no longer need to be mangled * Fix tests for VS older that VS2019 - __pragma in macro behavior seems inconsistent - just do things manually for now * Replace master with better wording (fastbuild#756) * Replace master with better wording * Force rebuild * Replace blacklist with better wording (fastbuild#757) * Added “AllowResponseFile” setting. (fastbuild#501) Co-authored-by: ffulin <franta.fulin@gmail.com> * Post-integration cleanup: AllowResponseFiles - add documentation - fix unreachable code - update integraiton/markup files - increment database version * Fix typo in test bff [skip ci] * [Improvement] ObjectLists can be chained using .CompilerInputObjectLists * Fix memory leak in cache test * Code style cleanup: - remove use of auto - replace some manual loops with range-based for - add some missing const * LZ4: Remove .gitignore and Makefile * [Fix][Windows] Prevent deadlock in FBuildWorker on shutdown when it's sent a WM_QUIT by another application * [Improvement] VCXProject and XCodeProject include .editorconfig files by default [Fix] Fix documentation errors for VCXProject and XCodeProject * Update Visual Studio 2019 to 14.26.28801 (16.6.4) * Emit monitor events for EXEC_NODE (fastbuild#768) * [WIP] Add new `docs/troubleshooting/cache.html` file to describe `cache debugging` strategies (fastbuild#729) This document is a direct result of the discussions and debugging here: fastbuild#727 NOTE: There remain some `TODO`(s) and unfinished sections to this file. I don't have all of the knowledge required to finish these sections properly. Hopefully @ffulin can fill in the blanks here. * Update `docs/documentation.html` to add a hyperlink to this new file. Signed-off-by: Layla <layla@insightfulvr.com> * Further improvements to Cache troubleshooting documentation * Documentation styling improvements and fixes * Add .SourceMapping_Experimental to Compiler node (fastbuild#769) * Post integration: Cleanup SourceMapping_Experimental - minor code style fixes - enable test on Windows with Clang - disable distributed compilation temporartily when used and document that * [Fix] ForEach correctly handles comma separator in parallel loops * Update FASTBuild to use VS2019 16.7.0 (14.27.29110) - fix various newly surfaced compiler warnings * Fix *ExcludePattern inconsistencies with slash direction [Fix] CSAssembly() handles forward and backslashes correctly on all platforms in .CompilerInputExcludePattern [Fix] Exec() handles forward and backslashes correctly on all platforms in .ExecInputExcludePattern [Fix] ObjectList() handles forward and backslashes correctly on all platforms in .CompilerInputExcludePattern [Fix] Test() handles forward and backslashes correctly on all platforms in .TestInputExcludePattern * Generate integration files for recently added SourceMapping_Experimental [skip ci] * Response file improvements [Improvement] Compiler() response file use can be forced with .ForceResponseFile [Improvement] Library() response file use can be forced with .LibrarianForceResponseFile [Improvement] Executable()/DLL() response file use can be forced with .LinkerForceResponseFile [Improvement][Linux] Command line limits (128 KiB) are enforced on Linux to make problems more obvious [Fix][Linux] Various "CanUseResponseFile" options are respected on Linux * Possible fix for file permissions problems on CI Uniquify response file names over time as a possible work around for permissions isuses on some machines - files reliably fail to open on the CI for some reason when re-used in rapid succession - this may be related to security software of some kind * Increase timeout on Response file permissions workaround for Windows * Improve output on failing assertion * Enhance assertion message further * Fix Args test - prevent tests executing code that relies on WorkerThread having previously created directories - revert some changes made during troubleshooting this issue * [Fix] Fix a failure path writing response files that was not stopping compilation correctly * Update documentation for ForceResponseFile options [skip ci] * v1.02 * Add ListDependencies() function to generate dependency text files for UE4 (fastbuild#760) * Post integration: ListDependencies - Emit output messages consistently with other nodes - Emit error codes in failure messages if output file writing fails - minor code style fixes * Apple Silicon configs (fastbuild#774) * Post integratoin: Support for Apply silicon natgive binaries - use longer sleep in TestTimer::Validate only on OSX ARM - minor code style tweaks * Travis CI: Update tests to use v1.02 binaries * Small clean-up, remove LibraryNode::GetLibrarian and use m_Librarian instead of GetName (fastbuild#710) Co-authored-by: Francisco Facioni <francisco.facioni@hawkeyeinnovations.com> Co-authored-by: ffulin <franta.fulin@gmail.com> * Improve include path extraction in ProjectGeneratorBase (fastbuild#701) * Add `/imsvc` option supported by clang-cl. * Add `-idirafter` option supported by GCC and Clang. * Extract include paths according to the priority of options that introduce them. Eg. path added via `-I` will be searched before the one that is added via `-isystem` even if it appears later on the command line. * Fix extraction of the option argument from `-isystem-afterDIR` which didn't work correctly due to a missing `break`. Previously `-afterDIR` was extracted in that case as the include path. * Add test for the order of include paths in generated project files. * Add all recognized options for specifying include paths to the documentation. Co-authored-by: ffulin <franta.fulin@gmail.com> * Post integration: Improved Intellisense/CodeSense inlude path extraction - fix various code style issues * Improve extraction of libraries from command line for GCC (fastbuild#702) * Support `-l:filename` syntax which allows to link a specific file. * Support `-Bstatic` and `-Bdynamic` options which control whether `-lfoo` checks for existence of a corresponding dynamic library file (`libfoo.so`) or not. * Change interpretation of `-l` options to match behavior of GNU ld - `-Bdynamic` is the default for systems that support dynamic linking, so for `-l` options we check for dynamic libraries first. * Post integration: linker command line dependency extraction fixes/improvements - code style cleanup * Add ability to manually specifiy workers via an environment variable. (fastbuild#689) E.g FASTBUILD_WORKERS=10.0.0.1;10.0.0.2;10.0.0.3 Co-authored-by: eclazi <blank> * Post integration: FASTBUILD_WORKERS environment variable - move logic from FBuild -> WorkerBrokerage - code style - add documentation - rename variabels and functions related to init of the brokerage for clarity * Rename AutoPtr -> UniquePtr * Remove some unnecessary includes * Fix BFFFuzzer compilation * Code cleanup - Add missing const - Remove some goto instances - Add missing ovveride keywords - Use constexpr in a few places - Add some missing default cases to switch statements * Code style fixes (Thanks to Joe Kirchoff) * Protocol: Fix compilation when TCP_DEBUG is enabled - an array of packet string names was out of date * MemDebug: Fix debug memory fill for remainder bytes - memory fill of remainder was wiritng pattern incorrectly - additionally simplify code slightly * Improve Static Analysis strictness for VS2019: - enable all static analysis checks and opt-out of those that cause errors - will may be able to keep more enabled in the future * Fix OSX and Linux compilation * [Improvement] MSVC linker failure LNK1201 is automatically retried to work around VS2019 bug * Tidy VS2019 static analysis flags * [Fix] Visual Studio solution header for VS2019 so version selector works correctly * Update brokerage file to use ip address instead of host name (fastbuild#770) * Add domain name to hostname results * Update brokerage file to include ip address Brokerage filename now contains ip address in addition to the host name Connection treats host string as a + delimited array and will attempt resolving each entry to an ip address * fix ::getdomainname check * Change all brokerage timers to use seconds for consistency * whitespace * Continue attempting connection to additional worker IPs * Ensure hostname is updated in WorkerBrokerage if changed Get hostname from WorkerBrokerage and pass to SetStatus() instead of also retrieving in WorkerWindow Handle empty strings in SetStatus so display text is formatted correctly if host name hasn't been acquired yet in WorkerBrokerage * Use ipaddress for brokerage filename If ip address is not yet available, use host name instead In addition to host name, write domain name and FQDN to brokerage file if domain name is not blank Add Network:GetDomainName() Reset TCPConnectionPool::Connect() to previous functionality where only one hostname/ip address is accepted instead of the + delimited string Reset GetHostName to previous functionality where only host name is returned, instead of FQDN Reset PROTOCOL_VERSION to 21 * revert TCPConnectionPool.cpp to before PR * revert WorkerBrokerage::FindWorkers() to before PR * Fix linux\apple typo Co-authored-by: ffulin <franta.fulin@gmail.com> * Post-integration cleanup: Worker brokerage by IP - various style fixes * [New] -wsl option can forward to Windows Subsystem for Linux in Visual Studio * Rename FBuild.exe project/library from FBuildApp to FBuild * BFF parses fixes for User Functions: [Fix] Escaped characters in string literal args to User Functions were not escaped correctly [Fix] $ Subsititutions in string literal args to User Functions were not handled in the correct scope * [Fix] #if directives were not correctly handled when used inside list declarations (and possibly other cases) - change how the EndOfFile token is managed when parsing if directives to avoid edge cases. EndOfFile now never appears incorrectly in the middle of a stream, which would cause inconsistent behavior depending on what was being parsed when we encountered an ifdef * Add missing newline character in ListDependenciesNode::EmitOutputMessage() log (fastbuild#796) Co-authored-by: Maxime Houlier <maxime.houlier@dont-nod.com> * [Fix] Fix features documentation typo (Thanks to Aleksey Kladov) * Simplify Visual Studio project generation with some bff helper functions * Compile with GCC9 and Clang10 on Linux by default * Core: adapt to Aarch64 on Linux (ARM64) (fastbuild#792) * Post integration: ARM Linux compatibility - minor whitespace fixes * Tidy up FASTBuild VS projects: - consolidate use of helper function(s) - add solution items for miscellaneous files * Linux: Set thread stack size always above PTHREAD_STACK_MIN. (fastbuild#791) PTHREAD_STACK_MIN=131072 on Aarch64 on Raspberry Pi OS, which made thread creation with stackSize=65536 fail. * Post commit: Respect PTHREAD_STACK_MIN on OSX & Linux - minor code style fixes * [New] Support integration of FASTBuild, Windows Subsystem for Linux and Visual Studio [Note] DB version has changed: Rebuild will occur [New] Support integration of FASTBuild, Windows Subsystem for Linux and Visual Studio - add various project generation options to assist with compiling and debugging - add "-wsl" helper command line option to allow invocation of commands in WSL * [Improvement] Add explicit support for Clang on windows in clang-cl mode (Thanks to Lambert Clara) - add a new compiler famiily and auto-detect clang-cl use - this allows clang to be used in "cl" mode, while allowing us to handle some differences versus the real cl.exe * Fix dispatch re-entry check in Tracing::Callbacks::DispatchCallbacksDebugSpam() to allow further logs to be recorded. (fastbuild#789) This issue occured with PROTOCOL_DEBUG_ENABLED defined, which enables the use of DEBUGSPAM() to output protocol debug log, and was preventing any further logs. Co-authored-by: Maxime Houlier <maxime.houlier@dont-nod.com> * Fix bad merge causing incorrect WARNING: output * [Fix] Missing errors for preprocessors that write to stdout on failure * [Improvement] Add .Libraries2 to Executable and DLL which can be accessed with %1[1] * Add lz4 1.9.3 source (not yet used) - will be used soon. Submitting to keep in sync with canonical Perforce repo * Add config to build FASTBuild with Clang 11 on Windows (unused) - submitting to keep github in sync with canonical Perforce depot * v1.03 * Switch dev branch tests to use FASTBuild v1.03 binaries * Switch binaries back to v1.02 OSX version requires a newer OS install which will need to be resolved * Use FASTBuild v1.03 binaries for CI - OSX binaries have been rebuilt to correctly support older OSX versions * Ensure compilation on OSX targets OSX 10.7 * Fix various compilation warnings * Fix various clang warnings: - superfluous ; - unused macros - 0 (or NULL) used for pointer instead of nullptr - mismatched include casing - use of deprecated types - and others * Fix missing line endings in various test files * Various clang-cl fixes: [Fix] Unity fixup is applied when using Clang-cl with static analysis so errors are correctly reported [Fix] Compilation with Clang doesn't fail when output is empty (can occur with Clang-cl with empty files) [Fix] SourceMapping_Experimental is applied correcly when using Clang-cl [Fix] Distributed compilation with Clang-cl is auto disabled when using SourceMapping_Experimental (as remapping is currently not distributed) [Improvement] When using Clang with static analysis, results are written to output file if no output is written by Clang itself [Fix] Warnings are printed for remotely built objects when using Clang-cl even if "warning as errors" is off * Fix Clang11 config for building FASTBuild: - switch to clang-cl for Clang11 to make management of Windows headers etc simpler - using Clang 11 when building with Clang on Windows (but keep Clang 7 use for Travis CI) * Simplify LZ4 build config: - disable all warnings instead of trying to disable them manually - there are too many warnings and they differ between Clang versions as well as clang.exe vs clang-cl.exe * Cleanup clang configs used to build FASTBuild on Windows: - use clang-cl for all versions * Update Visual Studio 2019 version to 16.8.3 (14.28.29333) * [Improvement] Distributed compilation and cache performance improved slightly due to LZ4 update (1.9.3) - Compression speed for preprocessed data improved ~5% - Decompression speed for preprocessed data improved ~5% - Compression speed for object files improved 5% - Decompression speed for object files improved upto ~10% * Fix MemoryLeak tracker when running on Windows with Clang * -fixuperrorpaths fixes [Fix] -fixuperrorpaths is correctly applied to Clang warnings using (x,y) style line numbering [Fix][OSX][Linux] -fixuperrorpaths with VBCC doesn't change slash direction on Linux/OSX * [Improvement] Dependency database memory usage reduced slightly - save 24 bytes per node by avoiding wasted bytes due to padding and oversized enum types * Remote Thread tracking: - Track which thread is used to build a job on remote hosts. This will be used for further changes to improve build visualization - Change ThreadIndex to uint16_t to avoid any increase in Job class size * ProfileManager: Fix FormatU64 so it handles printing zero correctly * [Improvement] Network Protocol bandwidth usage reduced slightly [Note] Distributed compilation protocol has changed: please update workers [Improvement] Network Protocol bandwidth usage reduced slightly - coupling small optimization with returning info about the thread used to compile remotely which will be used for future functionality * Fix typo in changelog: Compiation -> Compilation * -fastcancel is on by default [Improvement] FastCancel is enabled by default and can be disabled with -nofastcancel [Improvement] Remove deprecated -nooutputbuffering (option hasn't done anything for a long time) * Array: EmplaceBack returns reference to newly created element * [New] -profile option emits fbuild_profile.json in Chrome tracing format, which can be used to visualize the build * Update fbuild.bash-completion script for recent changes * [Improvement] Distributed compilation scheduling latency reduced by upto 100ms in certain scenarios Also: - improve accuracy of -profile display for distributed jobs, particularly when a remote jobs is returned and we cancel the local race * Minor DB serialization optimizations [Note] DB version changed - rebuild will occur [Improvement] Build database read/write optimized slightly * [Improvement][Windows] Reduce CPU overhead of FBuild slightly, particularly with many CPU cores - (Windows only) Limit the MainThread semaphore signal count to 1. Since it processes all work every time it awakens, additional signals are redundant. * WorkerThread: Improve profile macro setup - capture lifetime of worker threads more accurately - add macro to show per-thread tmp dir creation - make thread names display better for system with 10 or more cpu cores * Node: Fix return of void * FBuildWorker CPU usage, UI latency and shutdown speed improvements [Improvement] FBuildWorker idle CPU usage reduced, particularly for systems with many cpu cores [Improvement] FBuildWorker shuts down more quickly due to improved thread management [Improvement][Windows] Avoid per-CPU core stall updating UI, which could also delay exiting (up to 100ms per core) * [Improvement] Jobs are scheduled in a more efficient order when using distributed compilation (both locally and remotely) [Improvement] Jobs are scheduled in a more efficient order when using distributed compilation (both locally and remotely) - Jobs start being preprocessed in order of recursive cost, but can complete in any order. - We now sort the distributable jobs (the second/compilation pass) to ensure the most expensive jobs are started first (whether handed out to workers, or completed locally) - NOTE: The m_DistributableJobs_Available queue order is reversed (compared to before) so jobs are now removed from the end. This allows the same sorting helper to be used as for the main queue. * [New] Dependencies can be exported in DOT format using -dot and -dotfull - this allows visualiation of dependencies in tools that support the DOT file format * Ass some profiling macros * LightCache improvements/fixes: [Improvement] LightCache use is no longer inhibited by unused includes using macros: #include SOME_MACRO [Fix] Fix typo in 'Could not resolve macro' LightCache message * [Fix] Executable and DLL '%1[1]' is now correctly expanded even if '%1[0]' is not used * Fix Compiler() example for VS2019 in the documentation * BuildProfiler: Ensure backslashes in args are escaped in JSON output - args can contain backslashes if passed on the command line * Scheduling improvements - ObjectList/Library [Improvement] Compilation ordering improved by remove unnecesary dependencies from ObjectList and Library [Note] DB version changed * [Fix] Output messages no longer missing when using LightCache with -cachewrite and no distribution * Update .gitignore to exclude linux binaries * Update copyright to 2021 * v1.04 - Version bump - Documentation updates Co-authored-by: ggnkua <ggn@atari.org> Co-authored-by: George Nakos <george.nakos@u-blox.com> Co-authored-by: Brian Burgin <bburgin@users.noreply.github.com> Co-authored-by: Axel Wefers <awefers@icloud.com> Co-authored-by: Joe Kirchoff <Kirchoff.Joseph.P@gmail.com> Co-authored-by: Layla <laylamah@gmail.com> Co-authored-by: Javier Blazquez <jblazquez@riotgames.com> Co-authored-by: poppolopoppo <phamtom.x@gmail.com> Co-authored-by: Francisco Facioni <fran6co@gmail.com> Co-authored-by: Francisco Facioni <francisco.facioni@hawkeyeinnovations.com> Co-authored-by: dummyunit <dummyunit@gmail.com> Co-authored-by: eclazi <joss@jossgray.net> Co-authored-by: Joe Kirchoff <jkirchoff@fb.com> Co-authored-by: mhoulier <m.houlier@gmail.com> Co-authored-by: Maxime Houlier <maxime.houlier@dont-nod.com> Co-authored-by: Nicolas <knos@free.fr>
Dev (fastbuild#801) * Travis CI: Update FASTBuild to v1.01 * Extending #if parsing with boolean && and || (fastbuild#733) * Extending #if parsing with boolean && and || * Fix build on gcc/g++ * Updated documentation * Extending #if parsing with boolean && and || * Fix build on gcc/g++ * Updated documentation * Apply && and || operator precedence (&& before ||) * Tokenizer now recognises two character operators && and ||, #if handler modified accordingly * Remove tests from if_directive.bff, add them into TestBFFParsing.cpp * Actually evaluate the first defined result during the || pass * Kludge tests a bit until better understood * Formatting and cleanup * Added errors for #if operator parsing, applied them to tests. Co-authored-by: George Nakos <george.nakos@u-blox.com> Co-authored-by: ffulin <franta.fulin@gmail.com> * Fix bad merge * Delete stale file [skip ci] * Post intergation: #if supports || and && - simplify logic - add tests for some additional cases - simplify tests - code style - add documentation for new errors - update documentation to reflect precedence rules * [Improvement] Increase thread stack size to avoid unexpected problems with cache plugins * CachePlugin improvements [Note] DB Version changed [Improvement] Cache plugin can be initialized via InitEx which provides details of operating mode (read/write/verbose) [Improvement] Settings exposes .CachePluginDLLConfig to all passing of custom config to cache plugins [Improvement][Windows] Cache plugin DLL function names no longer need to be mangled * Fix tests for VS older that VS2019 - __pragma in macro behavior seems inconsistent - just do things manually for now * Replace master with better wording (fastbuild#756) * Replace master with better wording * Force rebuild * Replace blacklist with better wording (fastbuild#757) * Added “AllowResponseFile” setting. (fastbuild#501) Co-authored-by: ffulin <franta.fulin@gmail.com> * Post-integration cleanup: AllowResponseFiles - add documentation - fix unreachable code - update integraiton/markup files - increment database version * Fix typo in test bff [skip ci] * [Improvement] ObjectLists can be chained using .CompilerInputObjectLists * Fix memory leak in cache test * Code style cleanup: - remove use of auto - replace some manual loops with range-based for - add some missing const * LZ4: Remove .gitignore and Makefile * [Fix][Windows] Prevent deadlock in FBuildWorker on shutdown when it's sent a WM_QUIT by another application * [Improvement] VCXProject and XCodeProject include .editorconfig files by default [Fix] Fix documentation errors for VCXProject and XCodeProject * Update Visual Studio 2019 to 14.26.28801 (16.6.4) * Emit monitor events for EXEC_NODE (fastbuild#768) * [WIP] Add new `docs/troubleshooting/cache.html` file to describe `cache debugging` strategies (fastbuild#729) This document is a direct result of the discussions and debugging here: fastbuild#727 NOTE: There remain some `TODO`(s) and unfinished sections to this file. I don't have all of the knowledge required to finish these sections properly. Hopefully @ffulin can fill in the blanks here. * Update `docs/documentation.html` to add a hyperlink to this new file. Signed-off-by: Layla <layla@insightfulvr.com> * Further improvements to Cache troubleshooting documentation * Documentation styling improvements and fixes * Add .SourceMapping_Experimental to Compiler node (fastbuild#769) * Post integration: Cleanup SourceMapping_Experimental - minor code style fixes - enable test on Windows with Clang - disable distributed compilation temporartily when used and document that * [Fix] ForEach correctly handles comma separator in parallel loops * Update FASTBuild to use VS2019 16.7.0 (14.27.29110) - fix various newly surfaced compiler warnings * Fix *ExcludePattern inconsistencies with slash direction [Fix] CSAssembly() handles forward and backslashes correctly on all platforms in .CompilerInputExcludePattern [Fix] Exec() handles forward and backslashes correctly on all platforms in .ExecInputExcludePattern [Fix] ObjectList() handles forward and backslashes correctly on all platforms in .CompilerInputExcludePattern [Fix] Test() handles forward and backslashes correctly on all platforms in .TestInputExcludePattern * Generate integration files for recently added SourceMapping_Experimental [skip ci] * Response file improvements [Improvement] Compiler() response file use can be forced with .ForceResponseFile [Improvement] Library() response file use can be forced with .LibrarianForceResponseFile [Improvement] Executable()/DLL() response file use can be forced with .LinkerForceResponseFile [Improvement][Linux] Command line limits (128 KiB) are enforced on Linux to make problems more obvious [Fix][Linux] Various "CanUseResponseFile" options are respected on Linux * Possible fix for file permissions problems on CI Uniquify response file names over time as a possible work around for permissions isuses on some machines - files reliably fail to open on the CI for some reason when re-used in rapid succession - this may be related to security software of some kind * Increase timeout on Response file permissions workaround for Windows * Improve output on failing assertion * Enhance assertion message further * Fix Args test - prevent tests executing code that relies on WorkerThread having previously created directories - revert some changes made during troubleshooting this issue * [Fix] Fix a failure path writing response files that was not stopping compilation correctly * Update documentation for ForceResponseFile options [skip ci] * v1.02 * Add ListDependencies() function to generate dependency text files for UE4 (fastbuild#760) * Post integration: ListDependencies - Emit output messages consistently with other nodes - Emit error codes in failure messages if output file writing fails - minor code style fixes * Apple Silicon configs (fastbuild#774) * Post integratoin: Support for Apply silicon natgive binaries - use longer sleep in TestTimer::Validate only on OSX ARM - minor code style tweaks * Travis CI: Update tests to use v1.02 binaries * Small clean-up, remove LibraryNode::GetLibrarian and use m_Librarian instead of GetName (fastbuild#710) Co-authored-by: Francisco Facioni <francisco.facioni@hawkeyeinnovations.com> Co-authored-by: ffulin <franta.fulin@gmail.com> * Improve include path extraction in ProjectGeneratorBase (fastbuild#701) * Add `/imsvc` option supported by clang-cl. * Add `-idirafter` option supported by GCC and Clang. * Extract include paths according to the priority of options that introduce them. Eg. path added via `-I` will be searched before the one that is added via `-isystem` even if it appears later on the command line. * Fix extraction of the option argument from `-isystem-afterDIR` which didn't work correctly due to a missing `break`. Previously `-afterDIR` was extracted in that case as the include path. * Add test for the order of include paths in generated project files. * Add all recognized options for specifying include paths to the documentation. Co-authored-by: ffulin <franta.fulin@gmail.com> * Post integration: Improved Intellisense/CodeSense inlude path extraction - fix various code style issues * Improve extraction of libraries from command line for GCC (fastbuild#702) * Support `-l:filename` syntax which allows to link a specific file. * Support `-Bstatic` and `-Bdynamic` options which control whether `-lfoo` checks for existence of a corresponding dynamic library file (`libfoo.so`) or not. * Change interpretation of `-l` options to match behavior of GNU ld - `-Bdynamic` is the default for systems that support dynamic linking, so for `-l` options we check for dynamic libraries first. * Post integration: linker command line dependency extraction fixes/improvements - code style cleanup * Add ability to manually specifiy workers via an environment variable. (fastbuild#689) E.g FASTBUILD_WORKERS=10.0.0.1;10.0.0.2;10.0.0.3 Co-authored-by: eclazi <blank> * Post integration: FASTBUILD_WORKERS environment variable - move logic from FBuild -> WorkerBrokerage - code style - add documentation - rename variabels and functions related to init of the brokerage for clarity * Rename AutoPtr -> UniquePtr * Remove some unnecessary includes * Fix BFFFuzzer compilation * Code cleanup - Add missing const - 10000 Remove some goto instances - Add missing ovveride keywords - Use constexpr in a few places - Add some missing default cases to switch statements * Code style fixes (Thanks to Joe Kirchoff) * Protocol: Fix compilation when TCP_DEBUG is enabled - an array of packet string names was out of date * MemDebug: Fix debug memory fill for remainder bytes - memory fill of remainder was wiritng pattern incorrectly - additionally simplify code slightly * Improve Static Analysis strictness for VS2019: - enable all static analysis checks and opt-out of those that cause errors - will may be able to keep more enabled in the future * Fix OSX and Linux compilation * [Improvement] MSVC linker failure LNK1201 is automatically retried to work around VS2019 bug * Tidy VS2019 static analysis flags * [Fix] Visual Studio solution header for VS2019 so version selector works correctly * Update brokerage file to use ip address instead of host name (fastbuild#770) * Add domain name to hostname results * Update brokerage file to include ip address Brokerage filename now contains ip address in addition to the host name Connection treats host string as a + delimited array and will attempt resolving each entry to an ip address * fix ::getdomainname check * Change all brokerage timers to use seconds for consistency * whitespace * Continue attempting connection to additional worker IPs * Ensure hostname is updated in WorkerBrokerage if changed Get hostname from WorkerBrokerage and pass to SetStatus() instead of also retrieving in WorkerWindow Handle empty strings in SetStatus so display text is formatted correctly if host name hasn't been acquired yet in WorkerBrokerage * Use ipaddress for brokerage filename If ip address is not yet available, use host name instead In addition to host name, write domain name and FQDN to brokerage file if domain name is not blank Add Network:GetDomainName() Reset TCPConnectionPool::Connect() to previous functionality where only one hostname/ip address is accepted instead of the + delimited string Reset GetHostName to previous functionality where only host name is returned, instead of FQDN Reset PROTOCOL_VERSION to 21 * revert TCPConnectionPool.cpp to before PR * revert WorkerBrokerage::FindWorkers() to before PR * Fix linux\apple typo Co-authored-by: ffulin <franta.fulin@gmail.com> * Post-integration cleanup: Worker brokerage by IP - various style fixes * [New] -wsl option can forward to Windows Subsystem for Linux in Visual Studio * Rename FBuild.exe project/library from FBuildApp to FBuild * BFF parses fixes for User Functions: [Fix] Escaped characters in string literal args to User Functions were not escaped correctly [Fix] $ Subsititutions in string literal args to User Functions were not handled in the correct scope * [Fix] #if directives were not correctly handled when used inside list declarations (and possibly other cases) - change how the EndOfFile token is managed when parsing if directives to avoid edge cases. EndOfFile now never appears incorrectly in the middle of a stream, which would cause inconsistent behavior depending on what was being parsed when we encountered an ifdef * Add missing newline character in ListDependenciesNode::EmitOutputMessage() log (fastbuild#796) Co-authored-by: Maxime Houlier <maxime.houlier@dont-nod.com> * [Fix] Fix features documentation typo (Thanks to Aleksey Kladov) * Simplify Visual Studio project generation with some bff helper functions * Compile with GCC9 and Clang10 on Linux by default * Core: adapt to Aarch64 on Linux (ARM64) (fastbuild#792) * Post integration: ARM Linux compatibility - minor whitespace fixes * Tidy up FASTBuild VS projects: - consolidate use of helper function(s) - add solution items for miscellaneous files * Linux: Set thread stack size always above PTHREAD_STACK_MIN. (fastbuild#791) PTHREAD_STACK_MIN=131072 on Aarch64 on Raspberry Pi OS, which made thread creation with stackSize=65536 fail. * Post commit: Respect PTHREAD_STACK_MIN on OSX & Linux - minor code style fixes * [New] Support integration of FASTBuild, Windows Subsystem for Linux and Visual Studio [Note] DB version has changed: Rebuild will occur [New] Support integration of FASTBuild, Windows Subsystem for Linux and Visual Studio - add various project generation options to assist with compiling and debugging - add "-wsl" helper command line option to allow invocation of commands in WSL * [Improvement] Add explicit support for Clang on windows in clang-cl mode (Thanks to Lambert Clara) - add a new compiler famiily and auto-detect clang-cl use - this allows clang to be used in "cl" mode, while allowing us to handle some differences versus the real cl.exe * Fix dispatch re-entry check in Tracing::Callbacks::DispatchCallbacksDebugSpam() to allow further logs to be recorded. (fastbuild#789) This issue occured with PROTOCOL_DEBUG_ENABLED defined, which enables the use of DEBUGSPAM() to output protocol debug log, and was preventing any further logs. Co-authored-by: Maxime Houlier <maxime.houlier@dont-nod.com> * Fix bad merge causing incorrect WARNING: output * [Fix] Missing errors for preprocessors that write to stdout on failure * [Improvement] Add .Libraries2 to Executable and DLL which can be accessed with %1[1] * Add lz4 1.9.3 source (not yet used) - will be used soon. Submitting to keep in sync with canonical Perforce repo * Add config to build FASTBuild with Clang 11 on Windows (unused) - submitting to keep github in sync with canonical Perforce depot * v1.03 Co-authored-by: ggnkua <ggn@atari.org> Co-authored-by: George Nakos <george.nakos@u-blox.com> Co-authored-by: Brian Burgin <bburgin@users.noreply.github.com> Co-authored-by: Axel Wefers <awefers@icloud.com> Co-authored-by: Joe Kirchoff <Kirchoff.Joseph.P@gmail.com> Co-authored-by: Layla <laylamah@gmail.com> Co-authored-by: Javier Blazquez <jblazquez@riotgames.com> Co-authored-by: poppolopoppo <phamtom.x@gmail.com> Co-authored-by: Francisco Facioni <fran6co@gmail.com> Co-authored-by: Francisco Facioni <francisco.facioni@hawkeyeinnovations.com> Co-authored-by: dummyunit <dummyunit@gmail.com> Co-authored-by: eclazi <joss@jossgray.net> Co-authored-by: Joe Kirchoff <jkirchoff@fb.com> Co-authored-by: mhoulier <m.houlier@gmail.com> Co-authored-by: Maxime Houlier <maxime.houlier@dont-nod.com> Co-authored-by: Nicolas <knos@free.fr>
v1.02 * Travis CI: Update FASTBuild to v1.01 * Extending #if parsing with boolean && and || (fastbuild#733) * Extending #if parsing with boolean && and || * Fix build on gcc/g++ * Updated documentation * Extending #if parsing with boolean && and || * Fix build on gcc/g++ * Updated documentation * Apply && and || operator precedence (&& before ||) * Tokenizer now recognises two character operators && and ||, #if handler modified accordingly * Remove tests from if_directive.bff, add them into TestBFFParsing.cpp * Actually evaluate the first defined result during the || pass * Kludge tests a bit until better understood * Formatting and cleanup * Added errors for #if operator parsing, applied them to tests. Co-authored-by: George Nakos <george.nakos@u-blox.com> Co-authored-by: ffulin <franta.fulin@gmail.com> * Fix bad merge * Delete stale file [skip ci] * Post intergation: #if supports || and && - simplify logic - add tests for some additional cases - simplify tests - code style - add documentation for new errors - update documentation to reflect precedence rules * [Improvement] Increase thread stack size to avoid unexpected problems with cache plugins * CachePlugin improvements [Note] DB Version changed [Improvement] Cache plugin can be initialized via InitEx which provides details of operating mode (read/write/verbose) [Improvement] Settings exposes .CachePluginDLLConfig to all passing of custom config to cache plugins [Improvement][Windows] Cache plugin DLL function names no longer need to be mangled * Fix tests for VS older that VS2019 - __pragma in macro behavior seems inconsistent - just do things manually for now * Replace master with better wording (fastbuild#756) * Replace master with better wording * Force rebuild * Replace blacklist with better wording (fastbuild#757) * Added “AllowResponseFile” setting. (fastbuild#501) Co-authored-by: ffulin <franta.fulin@gmail.com> * Post-integration cleanup: AllowResponseFiles - add documentation - fix unreachable code - update integraiton/markup files - increment database version * Fix typo in test bff [skip ci] * [Improvement] ObjectLists can be chained using .CompilerInputObjectLists * Fix memory leak in cache test * Code style cleanup: - remove use of auto - replace some manual loops with range-based for - add some missing const * LZ4: Remove .gitignore and Makefile * [Fix][Windows] Prevent deadlock in FBuildWorker on shutdown when it's sent a WM_QUIT by another application * [Improvement] VCXProject and XCodeProject include .editorconfig files by default [Fix] Fix documentation errors for VCXProject and XCodeProject * Update Visual Studio 2019 to 14.26.28801 (16.6.4) * Emit monitor events for EXEC_NODE (fastbuild#768) * [WIP] Add new `docs/troubleshooting/cache.html` file to describe `cache debugging` strategies (fastbuild#729) This document is a direct result of the discussions and debugging here: fastbuild#727 NOTE: There remain some `TODO`(s) and unfinished sections to this file. I don't have all of the knowledge required to finish these sections properly. Hopefully @ffulin can fill in the blanks here. * Update `docs/documentation.html` to add a hyperlink to this new file. Signed-off-by: Layla <layla@insightfulvr.com> * Further improvements to Cache troubleshooting documentation * Documentation styling improvements and fixes * Add .SourceMapping_Experimental to Compiler node (fastbuild#769) * Post integration: Cleanup SourceMapping_Experimental - minor code style fixes - enable test on Windows with Clang - disable distributed compilation temporartily when used and document that * [Fix] ForEach correctly handles comma separator in parallel loops * Update FASTBuild to use VS2019 16.7.0 (14.27.29110) - fix various newly surfaced compiler warnings * Fix *ExcludePattern inconsistencies with slash direction [Fix] CSAssembly() handles forward and backslashes correctly on all platforms in .CompilerInputExcludePattern [Fix] Exec() handles forward and backslashes correctly on all platforms in .ExecInputExcludePattern [Fix] ObjectList() handles forward and backslashes correctly on all platforms in .CompilerInputExcludePattern [Fix] Test() handles forward and backslashes correctly on all platforms in .TestInputExcludePattern * Generate integration files for recently added SourceMapping_Experimental [skip ci] * Response file improvements [Improvement] Compiler() response file use can be forced with .ForceResponseFile [Improvement] Library() response file use can be forced with .LibrarianForceResponseFile [Improvement] Executable()/DLL() response file use can be forced with .LinkerForceResponseFile [Improvement][Linux] Command line limits (128 KiB) are enforced on Linux to make problems more obvious [Fix][Linux] Various "CanUseResponseFile" options are respected on Linux * Possible fix for file permissions problems on CI Uniquify response file names over time as a possible work around for permissions isuses on some machines - files reliably fail to open on the CI for some reason when re-used in rapid succession - this may be related to security software of some kind * Increase timeout on Response file permissions workaround for Windows * Improve output on failing assertion * Enhance assertion message further * Fix Args test - prevent tests executing code that relies on WorkerThread having previously created directories - revert some changes made during troubleshooting this issue * [Fix] Fix a failure path writing response files that was not stopping compilation correctly * Update documentation for ForceResponseFile options [skip ci] * v1.02 Co-authored-by: ggnkua <ggn@atari.org> Co-authored-by: George Nakos <george.nakos@u-blox.com> Co-authored-by: Brian Burgin <bburgin@users.noreply.github.com> Co-authored-by: Axel Wefers <awefers@icloud.com> Co-authored-by: Joe Kirchoff <Kirchoff.Joseph.P@gmail.com> Co-authored-by: Layla <laylamah@gmail.com> Co-authored-by: Javier Blazquez <jblazquez@riotgames.com>
PreviousNext