Process

April 2025 Update Blog Post

Process

SC::Process library allows spawning process across all supported platforms with many features, including I/O redirection and setting environment variables / starting directories.
This month however it has been gaining a new class called SC::ProcessFork that uses fork on Posix and RtlCloneUserProcess a very well hidden NT Kernel API that we can consider to be some sort of Windows fork.

A fork duplicates a parent process execution state, os handles and private memory.

Its semantics are quite different from platform to platform but on its most common denominator it can be used to carry on "background" operations on snapshots of current program memory. One relevant use case is serializing to disk or network a live, complex and large data structure. Without the fork the program should either:

Duplicate all the data, to snapshot it in a given instant, and keep it around for Async IO
Block program execution and write the live data-structure until all IO is finished

Fork avoids memory duplication because it will be shared through Copy On Write (COW) mechanisms.
COW ensures that un-modified duplicated memory pages will not occupy additional Physical RAM.

A pair of pipes makes it easy to do some coordination between parent and forked process.

This is not without caveats however for example:

Many API will just not work as expected on the forked process, especially on Windows
Limit API calls in forked process to console IO, network and file I/O (avoid GUI / Graphics)
All threads other than the current one will be suspended in child process (beware of deadlocks)
Create Sockets and FileDescriptors with Inheritable flags if you need them in fork process
Process deadlocks under Windows ARM64 / x86 emulation (use Process::IsWindowsEmulatedProcess)

So we're better using this feature only for very simple use cases, for example:

Foundation

This month the custom allocator system gains a new ability.
Anything SC::Segment based (Vector / String / VectorMap including their Inline variants) can be dumped and restored with a simple cast!
This is very useful for serialization purposes, for network protocols or state persistence to the disk, assuming there's no need to validate the data and/or to version it.

If data can evolve in different "versions" with added and removed field, you should probably look at SC::SerializationBinary.

Once everything is routed through an allocator, this is not difficult to do, but some effort has been spent to make it simple and easy.
Some debate on X has made it clear that this cannot be made fully UB safe under any existing C++ standard version so the usual caveats of "check what your compiler is doing" apply here.

Async

SC::Async library now supports vectorized writes.

This is the ability to atomically write multiple slices of memory with distinct start address and length.
It can be useful to reduce the number of syscalls required to execute the writes and also allows avoiding intermediate buffers to compose them in the proper order.

On posix this uses writev for both sockets and files.
On Windows WSASend can handle sockets in the same way, but WriteFile must be called multiple times in sequence for files.

SC::Async unifies such platform differences providing a simpler experience for library users.

Some other changes also have been done, centralizing AsyncRequest start in AsyncEventLoop and unifying AsyncTask for all request types.
These changes have been made to prepare for a larger feature that will land next month (surprise!).

Unfortunately some fields have been renamed, soft-breaking the API, but it shouldn't take more than a search and replace for library users to fix the errors.
Also the Async API will change again during next weeks so maybe it's a good idea to resume updating next month.

Tools

It's been a while since last refresh to SC::Tools!

SC::Tools are awesome, they just use Sane C++ Libraries to do the typical "scripting" that one would do in bash or python but with zero dependencies!

This month some dependencies have been updated, mainly clang-format, doxygen and 7zip.
Also the SC::Package tool that downloads / clones repos from inside SC::Build scripts is now using shallow cloning, bringing a noticeable speedup!

It's possible to now run SC::Format and build documentation and coverage on Linux instead of macOS only, so that the CI can just use any simple
Linux VM instead of a more complex and less available macOS one.

SC::Format uses clang-19 so make sure to use it if you ever plan to send an MR to the project (but the CI will error out immediately anyway, so you will find out very quickly).

The doxygen update was long overdue but every version of doxygen was plagued with some bug breaking the documentation.
The 1.12.0 version seems to work better, but if you see anything wrong please open an issue on github.

Tools: Add more descriptive messages for when git is missing
Tools: Enable running SC-Format on Linux
Tools: Fallback to use wget on Linux if curl is not available
Tools: Support shallow cloning of git repos
Tools: Update 7zr to 24.09
Tools: Update to clang-format 19
SCExample: Shallow clone dear-imgui and sokol
CI: Install 7zip to monitor its validity
[CI: Run clang-format and build documentation and coverage on linux](https://github.com/Pag...

April 2025 Update Blog Post

Process

SC::Process library allows spawning process across all supported platforms with many features, including I/O redirection and setting environment variables / starting directories.
This month however it has been gaining a new class called SC::ProcessFork that uses fork on Posix and RtlCloneUserProcess a very well hidden NT Kernel API that we can consider to be some sort of Windows fork.

A fork duplicates a parent process execution state, os handles and private memory.

Its semantics are quite different from platform to platform but on its most common denominator it can be used to carry on "background" operations on snapshots of current program memory. One relevant use case is serializing to disk or network a live, complex and large data structure. Without the fork the program should either:

Duplicate all the data, to snapshot it in a given instant, and keep it around for Async IO
Block program execution and write the live data-structure until all IO is finished

Fork avoids memory duplication because it will be shared through Copy On Write (COW) mechanisms.
COW ensures that un-modified duplicated memory pages will not occupy additional Physical RAM.

A pair of pipes makes it easy to do some coordination between parent and forked process.

This is not without caveats however for example:

Many API will just not work as expected on the forked process, especially on Windows
Limit API calls in forked process to console IO, network and file I/O (avoid GUI / Graphics)
All threads other than the current one will be suspended in child process (beware of deadlocks)
Create Sockets and FileDescriptors with Inheritable flags if you need them in fork process
Process deadlocks under Windows ARM64 / x86 emulation (use Process::IsWindowsEmulatedProcess)

So we're better using this feature only for very simple use cases.

Foundation

This month the custom allocator system gains a new ability.
Anything SC::Segment based (Vector / String / VectorMap including their Inline variants) can be dumped and restored with a simple cast!
This is very useful for serialization purposes, for network protocols or state persistence to the disk, assuming there's no need to validate the data and/or to version it.

If data can evolve in different "versions" with added and removed field, you should probably look at SC::SerializationBinary.

Once everything is routed through an allocator, this is not difficult to do, but some effort has been spent to make it simple and easy.
Some debate on X has made it clear that this cannot be made fully UB safe under any existing C++ standard version so the usual caveats of "check what your compiler is doing" apply here.

Async

SC::Async library now supports vectorized writes.

This is the ability to atomically write multiple slices of memory with distinct start address and length.
It can be useful to reduce the number of syscalls required to execute the writes and also allows avoiding intermediate buffers to compose them in the proper order.

On posix this uses writev for both sockets and files.
On Windows WSASend can handle sockets in the same way, but WriteFile must be called multiple times in sequence for files.

SC::Async unifies such platform differences providing a simpler experience for library users.

Some other changes also have been done, centralizing AsyncRequest start in AsyncEventLoop and unifying AsyncTask for all request types.
These changes have been made to prepare for a larger feature that will land next month (surprise!).

Unfortunately some fields have been renamed, soft-breaking the API, but it shouldn't take more than a search and replace for library users to fix the errors.
Also the Async API will change again during next weeks so maybe it's a good idea to resume updating next month.

Tools

It's been a while since last refresh to SC::Tools!

SC::Tools are awesome, they just use Sane C++ Libraries to do the typical "scripting" that one would do in bash or python but with zero dependencies!

This month some dependencies have been updated, mainly clang-format, doxygen and 7zip.
Also the SC::Package tool that downloads / clones repos from inside SC::Build scripts is now using shallow cloning, bringing a noticeable speedup!

It's possible to now run SC::Format and build documentation and coverage on Linux instead of macOS only, so that the CI can just use any simple
Linux VM instead of a more complex and less available macOS one.

SC::Format uses clang-19 so make sure to use it if you ever plan to send an MR to the project (but the CI will error out immediately anyway, so you will find out very quickly).

The doxygen update was long overdue but every version of doxygen was plagued with some bug breaking the documentation.
The 1.12.0 version seems to work better, but if you see anything wrong please open an issue on github.

Build

SC::Build has gained per-file include paths variable expansion for Visual Studio Projects and coverage computation on Linux.
Some features like nostdlib++ have been disabled on Linux because they need more love to properly support all combinations of it with the sanitizers.

Build: Do not use nostdlib++ on Linux
Build: Expand per-file include paths on VS Projects
[Build: Suppo...

February 2025 Update Blog Post

Containers

Re-writing Containers...what could be going wrong?

Sane C++ Libraries tag-line is "Platform Abstraction Libraries".
The main reason for that is that the project doesn't try or want to be a STL replacement.
There is an hard requirement not to depend on the Standard C++ Library because the standard totally disrespects one of the fundamental pillars of the project, namely Fast Compile Times.

In general STL-like containers may not be the best or most efficient abstraction in many cases.
Using them (or not) it's really an application choice that should not dictated by a Platform Abstraction library.

A lot of effort is often spent avoiding or at least reducing Containers usage in other Sane C++ Libraries API.
The problem is that Vector<T> containers were used both by Strings library and just as Buffers for File and other libraries.

For this reason I've decided to re-write the entire set of Containers with a few objectives

Cleaning-up the quite messy and verbose code for Vector<T> and SmallVector<T> and Array<T,N>
Provide a base implementation that works for char buffers but concise enough to be included in Foundations library.
Share as much code as possible to implement Vector<T> and SmallVector<T> that are still in Containers
Create a byte buffer implementation that should not leak in the headers

The results are quite satisfying, there is now a Buffer and SmallBuffer<N> that replace all Vector<char> usages and are implemented in Foundation.cpp file.
All the other Vector-like containers share most of the code and they're delivered as header only library as one can expect from a templated library.

Probably a next step could be evolving them to use custom memory allocators and arenas, but that will happen maybe in some future update.

This is the detailed list of commits:

An these are the commits were some dependencies from Containers have been removed

SC::Foundation

Changes in SC::Foundation are connected to the ones just described in Containers, because Segment class and Buffer plus SmallBuffer are all defined there.
Two classes, specifically TaggedUnion and TaggedMap have been moved to SC::FoundationExtra.
The reason is that they're not used by any other library and feel a little bit too Modern C++ to deserve a place in Sane C++ Libraries.

SC::Build

SC::Build is the self-hosted build system used by Sane C++ Libraries to generate test and example projects.
It's not needed to use the libraries, as they need no build system at all, but it's used when developing the libraries.

It's not ready for general use (yet!), but it's progressing towards getting there someday.

In this month the build definition API has been cleaned up a little bit, with preference towards using methods to configure the build rather than filling the data structure.

A new more precise flags resolution / merging system properly allows to override compile or link flags in configuration, so that they have priority over the ones set per-project.
It's also possible to set a compile flags for a specific set of files, including disabling warnings for them.

Result buildTestProject(const Parameters& parameters, Project& project)
{
    project = {TargetType::ConsoleExecutable, TEST_PROJECT_NAME};

    // All relative paths are evaluated from this project root directory.
    project.setRootDirectory(parameters.directories.libraryDirectory.view());

    // Project Configurations
    project.addPresetConfiguration(Configuration::Preset::Debug, parameters);
    project.addPresetConfiguration(Configuration::Preset::Release, parameters);
    project.addPresetConfiguration(Configuration::Preset::DebugCoverage, parameters);

    // Defines
    // $(PROJECT_ROOT) expands to Project::setRootDirectory expressed relative to $(PROJECT_DIR)
    project.addDefines({"SC_LIBRARY_PATH=$(PROJECT_ROOT)", "SC_COMPILER_ENABLE_CONFIG=1"});

    // Includes
    project.addIncludePaths({
        ".",            // Libraries path (for PluginTest)
        "Tests/SCTest", // SCConfig.h path (enabled by SC_COMPILER_ENABLE_CONFIG == 1)
    });

    addSaneCppLibraries(project, parameters);
    project.addFiles("Tests/SCTest", "*.cpp"); // add all .cpp from SCTest directory
    project.addFiles("Tests/SCTest", "*.h");   // add all .h from SCTest directory
    project.addFiles("Tools", "SC-*.cpp");     // add all tools
    project.addFiles("Tools", "*.h");          // add tools headers
    project.addFiles("Tools", "*Test.cpp");    // add tools tests

    // This is a totally useless per-file define to test "per-file" flags SC::Build feature.
    SourceFiles specificFiles;
    // For testing purposes let's create a needlessly complex selection filter for "SC Spaces.cpp"
    specificFiles.addSelection("Tests/SCTest", "*.cpp");
    specificFiles.removeSelection("Tests/SCTest", "SCTest.cpp");
    // Add an useless define to be checked inside "SC Spaces.cpp" and "SCTest.cpp"
    specificFiles.compile.addDefines({"SC_SPACES_SPECIFIC_DEFINE=1"});
    specificFiles.compile.addIncludePaths({"../Directory With Spaces"});

    // For testing purposes disable some warnings caused in "SC Spaces.cpp"
    specificFiles.compile.disableWarnings({4100});                                 // MSVC only
    specificFiles.compile.disableWarnings({"unused-parameter"});                   // GCC and Clang
    specificFiles.compile.disableClangWarnings({"reserved-user-defined-literal"}); // Clang Only
    project.addSpecificFileFlags(specificFiles);

    return Result(true);
}

This is the detailed list of commits:

Minor Changes

And as always here is the list of random fixes scattered around the library!

This is the detailed list of commits for all minor changes:

Async: Remove variable length array in Linux backend
AsyncStreams: Disable 8000 tests requiring ZLib on Windows ARM64
CI: Compile also SCExample
Everywhere: Fix some spelling errors
Plugin: Add close method to PluginRegistry
[Plugin: Move StringHashFNV to PluginHash](ht...

January 2025 Update Blog Post

SC::Async

SC::Async has been the main focus of the month.

Simplification:

Remove one state throughout the system (Teardown)

Consistency:

Expose callback to signal when a request is fully stopped after issuing a AsyncRequest::stop().

Features:

Exclude specific request from active count (it will not keep the loop alive)
Enumerate all (non-internal) active and submitted async requests
Interrupt event loop even with active requests
Update loop time externally
Check request state (free / active / cancelling)

Stability:

Fix issues with AsyncWakeUp on io_uring

Timers (AsyncLoopTimeout) improvements:

Invoke timers consistently when one of them is being cancelled during another timer's callback.
Sort timers by expiration time (first) and insertion order (second)
Use Monotonic clock everywhere
Unify code on all backends (including io_uring)

Detailed List of commits:

SC::AsyncStreams

Some minimal work as been done to the SC::AsyncStreams library as well.

One video has been recorded Showing how to move the transform stream compression operation on a background thread, using AsyncLoopWork.
This video pauses (for now) the series of videos dedicated to SC::AsyncStreams.

Detailed List of commits:

SC::Foundation

SC::Foundation most notable change has been using a different approach in SC::Function.

We're now using a vtable-like approach that allows saving one entire pointer (!!) for each SC::Function instance.

The price to pay is the static initialization (that will require a mutex acquisition) and one more indirection, but it has been considered a good tradeoff.

Detailed List of commits:

SC::Time

The SC::Time library got user defined literals, some conversion between different units and differentiation between monotonic and realtime clocks.

A nasty bug regarding time normalization in SC::Time::HighResolutionCounter has been fixed too!

Detailed List of commits:

Minor changes

And these are some minor changes that don't have enough impact to deserve a dedicated comment.

Detailed List of commits:

Full Changelog: release/2024/12...release/2025/01

December 2024 Update Blog Post

SC::AsyncStreams

Most of the work has been focused on shaping the SC::AsyncStreams library.

Async streams got their own folder and the AsyncPipeline API has been improved so that it accept transform streams.
Transform streams are used to concurrently modify data read from a source, before it's being writte 8000 n to one or multiple destinations.
The sample use case has been compression through ZLIB library.
In order to keep Sane C++ Libraries free of build time dependencies, zlib is dynamically loaded on all major operating systems.
A nice trick has been figuring out that the .NET CLR on Windows ships zlib hidden in the clrcompression.dll, avoiding the need to deploy it.

It seems that the fixed pool initial design constraint, that forbids any dynamic allocation inside Async Streams is still working.
Some more work will need to be carried on to really prove if it works on a significant number of use cases!

Two Videos have been recorded while implementing the most significant portions of SC::AsyncTransformStream

Detailed List of commits:

00b4e95 Move Async Streams to a dedicated folder
17a3618 Improve documentation
2183984 Use threaded blocking IO for file operations in test
21a0c92 Add an async zlib transform stream
21c8605 Replace Pipe with a Span of AsyncWritableStream
387cc1f Add method to remove all listeners bound to a specific class instance
402a214 Fix pause handling inside AsyncRequestReadableStream
70a4f92 Allow extending writable ending state until needed
741bf98 Add unpipe to remove listeners added by pipe
80b565e Add CircularQueue::pushFront
87e7652 Make AsyncPipeline end writable sinks when readable source ends
8a3c543 Add errors listener directly on the AsyncPipeline in the test
ad6a2b5 Add ZLibAPI and ZLibStream
c7c6714 Add Event::removeListener member function overload
d74291a Implement write resuming and pushing a buffer to the top of the write queue
de7e410 Add auto-close descriptor for request streams
df25bbf Add AsyncTransformStream and a simple synchronous zlib stream
ec5d68d Add AsyncPipeline::pipe to validate requested pipeline

SC::Async

SC::Async gets its usual set of monthly minimal improvements.
The most significant one is change in API when using thread pools, so that they can be set once and re-used over multiple start of the same request.

Detailed List of commits:

898a775 Require setting thread-pool before start for AsyncLoopWork
90fca39 Allow setting request thread pool and task before start
9cd6b84 Avoid leaking link to next element in ThreadSafeLinkedList
b05854d Set AsyncRequest to State::Free after stop()

SC::Foundation

SC::AsyncFoundation gets a few new classes to represent read-only and read/write strings without needing to include the entire SC::String library.

1fb1bf3 Fix Span::reinterpret_as_array_of const correctness
245cb38 Add method to check if a Function is bound to a specific class instance
9ba637d Add SpanStringView and SpanString
a16248d Add Span::sliceFromStartUntil and fix Span::equals
ae21ddc Fix Span::reinterpret_as_array_of

Refactoring

Changes in SC::AsyncFoundation have been carried on to reduce header bloat and inter-dependencies between libraries.
More specifically both File and Socket library got this treatment, so that Socket doesn't depend on SC::String library at all anymore, and SC::File now has a StringView-free header (FileDescriptor.h) that can be included without bringing any StringView dependency

a2f67be Split all String and Vector related functions out of FileDescriptor
77d2166 Split SocketDescriptor header

Minor changes

146608c Cleanup README.md by referencing relevant blogs/videos inside each library
b0154f3 Add some Quick Sheets and update README.md
a7aad86 Add function to obtain plugins to reload after a file is modified

Full Changelog: release/2024/11...release/2024/12

This update brings a new functionality: Async Streams

November 2024 Update Blog Post

Async Streams

The SC::Async library has been extended with the addition of Async Streams!

Async Streams are largely inspired by node.js Streams, a very powerful tool to process large amounts of data in parallel.

The basic idea about an async stream is to create a Source / Sink abstraction (also called Readable and Writable) and process small buffers of data at time.

The state machine that coordinates this interaction handles data buffering and more importantly handles also back-pressure, that means:

Pausing the readable stream when a connected writable stream cannot process data fast enough
Resuming the readable stream when a connected writable stream is finally able to receive more data

By implementing streams on top of async operations it's possible to run many of them concurrently very efficiently.
When properly implemented for example an async pipeline can concurrently read from disk, write to a socket while compressing data.
For now only Readable / Writable File and Socket streams have been implemented, but Async Transform Streams (for compression) will be next!

Most notable differences with node.js streams are for now:

No allocation (designed to work inside user-provided list of buffers)
No object mode
Fixed Layout to create data pipelines (AsyncPipeline)
onData support only (no readable event)

Async Streams are for now in 🟥 Draft state.
It's also possible that its API will evolve a little bit to be less verbose and there is also lack of nice examples, aside from the tests.

It's better waiting for it to become stable before doing anything significant with it.

Some changes and fixes in the SC::Async library
have been made necessary to support Async Streams.

339f291 Async: Add AsyncReadableStream
11a2b92 Async: Add AsyncWritableStream
9d9d550 Async: Draft AsyncPipeline and AsyncRequest{Readable | Writable} Stream
fc97464 Async: Extend AsyncRequest{Readable | Writable}Stream to Socket Send/Receive
46174b0 Async: Make AsyncPipeline and AsyncRequest{Readable | Writable}Stream public
15ec185 Async: Make AsyncSocket{Send | Receive} buffer and handle public
a6bdc11 Async: Rename AsyncPipeline::Sink to AsyncPipeline::Pipe
3c78017 Async: Reorganize AsyncTest for clarity
c251205 Async: Use offsets only if explicitly set in AsyncFile{Read | Write}

Others

And just like every update, a bunch of fixes and improvements to all libraries have been committed.
Nothing specific stands out this update, but for completeness this is the list of related commits:

9347ca6 Build: Bypass VMWare hgfs issue setting wrong modified time for new files
a075702 Build: Support linking system libraries in XCode
a572d89 Containers: Fix compile errors under latest MSVC
ee4947b Documentation: Enable warnings as errors
596f052 Documentation: Update README.md with latest videos
b077888 Foundation: Add equality operator to Function
0ad8187 Foundation: Add HeapBuffer
c750871 Foundation: Add Span::equals
7f08864 Foundation: Add Span::get(int)
14f37f5 Foundation: Add StrongID
e80edc2 Hashing: Improve documentation
23e7b37 Meta: Ignore sync folder and icon files
fa9a48c Tools: Rebuild bootstrap when make fails

Full Changelog: release/2024/08...release/2024/11

August has been spent doing some first round of improvements to the SC::Http library.

August 2024 Update Blog Post

Http

August has been spent doing some first round of improvements to the HTTP library.
The HTTP library is still very incomplete and fragile, but it got to the level of being able serving a simple local html web site, through the new SC::HttpWebServer class.
The SC::HttpServer / SC::HttpWebServer classes are not solid enough to be used in real internet facing application but they're in better shape than they were the previous month.
SC::HttpClient is just a very incomplete stub and it currently exists to help testing the server classes.

9f4700f Http: Add HttpWebServer to serve files from a directory
4d3a5b4 Http: Hide HttpServer implementation details
6f03e3b Http: Improve HttpServer properly releasing unused sockets

SCExample

SCExample has now a second example called WebServer that can host a website on a specific port reading files from a local folder, all integrated with a GUI interface.
The example interface now has two new methods (initAsync / closeAsync) to let samples access the main SC::AsyncEventLoop of the application.
The async io event loop is integrated with gui event loop on main thread to make sure that all examples using SC::Async can see their completions dispatched on the main (gui) thread. This means that they can modify the same state variables used by the GUI without needing mutexes.

SCExample_05.mp4

15eb5ab SCExample: Add helper to use InputText with SC::String
0893d45 SCExample: Add initAsync and closeAsync example interface methods
3814380 SCExample: Add WebServer example

Async

Implementing the SC::HttpWebServer has been putting a little bit of stress on the SC::Async library, with a few new features, and some bug fixes and QOL improvements.

Features

SC::AsyncSocketReceive and SC::AsyncFileRead report End Of File and disconnected events on all platforms, signaling when they are running out of data.

Bugs (fixed)

SC::AsyncSocketWrite writes were not properly handled on Posix with large buffers (and now there's a test for that)
Timer callbacks were called multiple times on the io_uring backend

Quality of Life improvements

It's possible now to de-allocate any SC::AsyncRequest derived request inside its own callback or just re-configure them and start them again.
That should make it a lot easier handling more complex async data flows.

1d262b7 Async: Allow AsyncRequests to be deallocated inside their own user callback
3735432 Async: Do not invoke timers callbacks multiple times on io_uring backend
1cf9c2b Async: Handle partial writes in Posix backend
34cbfd6 Async: Mark AsyncRequest as free after Cancelling it
9b1b2d0 Async: Report disconnected on AsyncSocketReceive
58aded2 Async: Report end of file on AsyncFileRead
a7c64cc Async: Simplify reporting AsyncSocketReceive disconnected on posix

Others

And just like every month, a bunch of fixes and improvements to all libraries have been committed.
The most notable bug was in the SC::Process library, that was preventing proper stdout inheritance for console applications on Windows.
Github CI was missing all output of the cl.exe and link.exe msvc executables spawned by SC::Build during compilation.
Specifically the Win10/MSVC2019 ci job was failing without printing any error, making it really difficult to figure out how to fix it!

97a687f Console: Attach to parent console on Windows
809e2b9 Containers: Add methods to obtain arena capacity
fb6877c Containers: Split ArenaMapKey in a separate header
0c0a6b6 FileSystem: Report file size in new FileStat structure replacing FileTime
12b85ed Process: Hide child process console on Windows only if parent process has no console
924a797 SerializationBinary: Prefer using loadExact when loaded schema matches source schema
1783a3a SerializationText: Support bool data types for JSON serializer
177f005 Socket: Replace SocketClient::close calls with SocketDescriptor::close
2cd0850 Time: Expose month and day as strings in Time::Absolute::ParseResult

@jlaumon

The focus of the month has been building a nicer showcase for Plugin and Serialization libraries.

July 2024 Update Blog Post
July 2024 Update Video

SCExample

The hot-reload system of SCExample has been tuned so that each internal example is just an hot-reloaded Plugin.
It's now very convenient to iteratively build them and experiment!
A nice serialization example has been added to SCExample, based on a very well known piece of code from the official imgui demo.
State of the canvas control is saved to binary and json using the automatic serialization provided by the reflection system.
It's also showing a somewhat advanced usage of the Reflection system, that is how to wrap a "custom" vector implementation (ImVector)

SCExample_04.mp4

There is a video where I've been recording the most important bits of this development:
C++ Serialization and Reflection (with Hot-Reload) - Sane C++ Libraries [ep.25]

And as a last nicety, SCExample has been fully ported to iOS!
Hot-reload obviously requires root access to install clang compiler and a sysroot on iOS, so it will work only if you have an active jailbreak.

Also here I've been recording a video on finalizing porting SCExample to iOS.

Hot-Reload C++ on iOS - Sane C++ Libraries [ep.24]

85e6352 SCExample: Port to iOS
c839d17 SCExample: Show compile errors when hovering examples
641c927 SCExample: Refactoring hot-reload plugins as examples
b14ef5e SCExample: Add Serialization example

Serialization

While building the Serialization example, a few fixes have been made necessary.
The first has been supporting the bool data type that for some obscure reason was missing from the primitive types.
Some helpers to read and write the schema together with serialized data have been added to make it easy supporting versioned binary serialization.
Fields can be added or removed, and the serializer will still try to load data that can be converted assuming a matching order.
There is also some support for changing field types. For example a float field that is converted to int (or vice-versa) will receive a truncated value on deserialization.

fe203bb Reflection: Add vector manipulation methods to ExtendedTypeInfo
0419d74 Reflection: Support bool primitive type
f83bb78 SerializationBinary: Add writeWithSchema and loadVersionedWithSchema
03ef3a1 SerializationBinary: Reduce boilerplate needed to wrap a custom vector container
18d871c SerializationBinary: Support bool primitive type
8981495 SerializationText: Reduce boilerplate needed to wrap a custom vector container

Plugin

Plugin library has received a few improvements, like the ability to specify a path for the sysroot under clang (needed on iOS), and it's capturing compiler and linker outputs.
This last one allows showing compile errors in the plugin host app, that is a nice addition to SCExample.
Finally some dynamic export clauses have been added to many libraries in order for Plugins to find symbols in the host application that is loading them.
This will for sure increase the executable size as the linker will not be able to trim such classes as "unused" code, even if they're not referenced by any Plugin.
Probably some macro to disable this "export by default" behavior will be added for anyone that wants to use Sane C++ Libraries without the Plugin Library or without needing to use Sane C++ Libraries type across dynamic library boundaries.

e6d21a8 Plugin: Allow overriding isysroot path
61b8928 Plugin: Capture compiler and linker output
1ffb874 Everywhere: Add export clause for use across Plugin boundaries

Build

Build system has received some improvements too, supporting generation of iOS project and application icon for Xcode app bundle.
There are still many missing features and too many hardcoded defaults but I feel that the library deserves more than a Draft status.
For this reason I've been promoting it to MVP status: maybe this will become an incentive to add more features to make it usable!

d865cc4 Build: Add iOS storyboard generation
ca0fb7b Build: Elevate to MVP status
6551c1e Build: Support application icon in Xcode app bundles
70d32b7 Build: Support iOS

Contributions

This month Jeremy Laumon has been adding natvis support for Arrays!
415dc32 DebugVisualizers: Add natvis for SegmentedItems and Array

Others

And then the usual amount of monthly fixes!
I can never stress how important is to use the libraries even just in SCExample, to trigger issues and identify missing bits.

050bad6 Strings: Export String and some SmallString for use across Plugin boundaries
1d48530 Containers: Fix some incorrect Array member functions
20f2dd9 FileSystemWatcher: Add internal header with FSEvents declarations for iOS
54fb28b Documentation: Improve README and fix descriptions of some libraries
5a72dcf FileSystem: Add read and write overloads for uint8_t
915367a Strings: Fix doubles parsing
a176a3e Everywhere: Add lifetime bound attribute for StringViews and Spans
e3121af Foundation: Add Span::reinterpret_as_array_of()
e5475af Tools: Use Github to download doxygen releases

What's Changed

DebugVisualizers: Added natvis for SegmentedItems and Array by @jlaumon in #20

New Contributors

@jlaumon made their first contribution in #20

Full Changelog: release/2024/06...release/2024/07

June 2024 Update Blog

The month has been spent mainly building SCExample, a small GUI example app to showcase some Libraries (mainly SC::Async / SC::Plugin / SC::FileSystemWatcher).

SCExample

SCExample is a small GUI application based on sokol_app / sokol_gfx libraries, providing window abstraction and graphics api abstraction and dear-imgui for the UI.

Building this application has been mostly documented in the following YouTube videos:

First step has been putting together a pure cross platform GUI application (win/linux/macos), making sure to pause the render loop in absence of user inputs:
Pause Immediate Mode UI - Save CPU Time

Second step has been integrating the GUI event loop with the SC::Async I/O library (always cross-platform)
Add Async IO to Immediate Mode GUI

And lastly showcasing the integration of SC::FileSystemWatcher and SC::Plugin to watch source files that are being hot-reloaded as plugins (dynamic libraries).
Also in this case the cross-platform aspect has been preserved.

Hot-Reload dear imgui

This is the list of SCExample related commits:
6f29135 SCExample: Create (sokol+dear imgui) app re-drawing on user input only
4c72b42 SCExample: Display number of reloads and last load time of hot-reloaded plugins
96e6df4 SCExample: Implement simple hot-reload to showcase Plugin and FileSystemWatcher libraries
5be2d02 SCExample: Integrate SC::AsyncEventLoop by using SC::AsyncEventLoopMonitor
947f8a7 SCExample: Transform floating window into a toolbar

SC::Async

A small but useful addition to the SC::Async library has been the SC::AsyncEventLoopMonitor class.
This class wraps most of the required machinery to integrate SC::Async event loop in an application that has a different main event loop (GUI or IO)

784ffcf Async: Add AsyncEventLoopMonitor to poll the event loop from a background thread

SC::Plugin

SC::Plugin library has been receiving some love, with the addition of a queryInterface-like mechanism helping to define contracts between plugins and the host application.

It's possible to control linking libc / libc++, adding some custom include directories and disabling exceptions.
SC::Plugin now monitors CFLAGS and LDFLAGS environment variables (if defined) of the host environment, to allow linking the correct sysroot on posix systems.
Some fixes too have been needed on macOS where duplicated plugin symbols from different compiled .dylibs can wrongly increment their refcount, preventing from unloading them.

d6816b1 Plugin: Add PluginCompilerEnvironment to intercept and use CFLAGS and LDFLAGS
94ad0d9 Plugin: Add PluginDynamicLibrary::queryInterface
efc4863 Plugin: Add PluginSysroot and build options to allow linking libc and libc++
64ee726 Plugin: Allow multiple include paths
9466040 Plugin: Change appendDefinitions into replaceDefinitions
72354c8 Plugin: Disable exceptions under MSVC
8ad2aca Plugin: Hide Plugin symbols by default on macOS
f7dfa22 Plugin: Track number of reloads and last load time for all plugins

SC::FileSystemWatcher

Using SC::FileSystemWatcher inside SC::Example has exposed some usability issues and bugs (mainly repeated notifications), that have been promptly fixed.

05c0edd FileSystemWatcher: Filter repeated notifications on macOS
ffada00 FileSystemWatcher: Improve API usability

SC::Build

SC::Build can now generate Xcode projects for gui applications creating app bundles on macOS for the Xcode backend.
The configure phase will download required dependencies (like sokol or dear-imgui for SCExample).
This is not ideal as it requires internet connection, so it will be probably be made optional in some future update.

It can now build from paths with spaces (even when using the Makefile backend...) and generates Xcode projects avoiding any warning.
Some of the fixes have ramifications also on the SC::Tools that bootstraps SC::Build.

2be7d39 Build: Collapse all non-apple TARGET_OS to just linux
931fe7c Build: Escape quotes in Make and Xcode backends
df3eae8 Build: Fix generation of compile_commands.json file
e6acc3d Build: Fix Makefile force clean on platform specific makefiles
b225291 Build: Generate makefiles supporting building from paths with spaces
f711566 Build: Improve Xcode generator not to produce warnings under Xcode 15
0214cdb Build: Support creating app bundles on Xcode

b2e3cef Tools: Improve Makefile
0b802ba Tools: Support building a tool from path containing spaces

Others

And just like every month, working on SCExample has generated a number of minor fixes, API improvements / refactoring.
All of them have been proudly committed to the main branch!

30f07bd Containers: Fix bug in VectorMap::remove

03b81f6 FileSystem: Add writeStringAppend

95d90ab Foundation: Add Host Operating System detection

4d9d9ae Process: Avoid creating windows when spawning a new process
a7859c5 Process: Improve ProcessTest compatibility
03cd51d Process: Rename StringsTable to StringsArena

1f33202 Reflection: Disable SC_REFLECT_AUTOMATIC on older clang versions
a941264 Reflection: Fix compile error due to a GCC bug

e230df7 Strings: Add StringHashFNV
1c0fd89 Strings: Do not let StringBuilder leave unterminated string when format fails

fd24695 Time: Make HighResolutionCounter::snap return reference to HighResolutionCounter itself

1b50cb3 Documentation: Add SCExample to the Examples page
32181ff Documentation: Update README with latest video
07591f1 Documentation: Update README with May 2024 blog post
476fbf8 Documentation: Update SCExample documentation

Full Changelog: release/2024/05...release/2024/06

@silent-tech

This has been a month of steady improvements, with no big feature added.

Make sure to check out the update blog post:
May 2024 Update Blog

Async Library has been receiving some fixes and a useful refactoring.

AsyncEventLoop::runOnce has been split into three methods, separating request submission, polling for new events and completion callbacks dispatch.
This allows submitting request from a "main" thread, that already has an event loop, blocking to poll for changes on a separate thread, and dispatching the callbacks for events received again on the same thread used for submission.
The main use of this would be integrating an AsyncEventLoop with another event loop from another library that already "owns" the application main thread, including a GUI event loop.
Loop time that is now being updated more consistently both when using runOnce() and runNoWait()
Kernel synchronization was failing under Windows 10.
The bug has been fixed and now Windows 10 is now being tested in the CI to avoid future regressions.
I've also been adding a new "phase" to handle re-activation that allows some requests to skip going through the submission queue, by just keeping it active.
Re-activation happens when inside a callback for some Async Request user calls reactivateRequest(true).
New code handles re-activation more correctly as on some backends (kqueue, epoll), where the request is always active once it has been started.
As submissions are processed on the "next" event loop run, it can happen that such requests are "triggered" by the kernel while they were still in submission queue.
This event breaks many assumptions made by the (kinda complex) state machine coordinated by AsyncEventLoop.

0526b5c Async: Split runOnce into submitRequests, blockingPoll and dispatchCompletions
069b4bd Async: More precise handling of GetQueuedCompletionStatusEx on Windows 10
138d1d8 Async: Allow re-activation of AsyncLoopTimeout
3d48f77 Async: Add ReactivateAsyncPhase to allow skipping submission on reactivation
af1e0f0 Async: Hide KernelQueue and KernelEvents inside Internal
b7623c5 Async: Handle different GetQueuedCompletionStatusEx behavior on Windows 10
e6a5704 Async: Update time consistently both in NoWait and ForcedForwardProgress modes

Build Library has also been receiving some love, increasing its capability while simplifying SC-build.cpp scripts:

Support generating VS2019 projects (needed by the Windows10 Github runner)
Support Objective-C or Objective-C++ files
Support generating multiple projects
Support libraries to link (Makefile backend)
Support creating non-console applications (Windows)
Consistently handle all paths (include, libraries etc) as relative to project root

2860888 Build: Generate one solution per project on Visual Studio backend
2d63b73 Build: Specify target name when invoking Makefile
35f5fc5 Build: Avoid Makefile warnings on intermediates and outputs directory creation
5879951 Build: Support Objective-C and Objective-C++ files
9f195b3 Build: Generate platform specific Makefile
a123422 Build: Express absolute include paths as relative to project dir
ad9f0ee Build: Add flag to generate Visual Studio projects using Windows subsystem
b90f05c Build: Allow selecting target for compile, build or run action
c9dd352 Build: Simplify SC-Build with more defaults
cab21d6 Build: Support link libraries on Makefile backend
df488cd Build: Simplify usage of relative paths in includes and defines
f0b3043 Build: Add basic multi-projects support

The CI has been simplified, consisting now of three github workflow file

windows (windows 10 + windows 11)
posix (linux + macOS)
documentation and coverage (macOS)

The GitHub CI matrix feature has been used to test multiple platforms and configurations.
On all platforms both Debug and Release configurations are now being tested.
Additionally also Windows 10 has been added to the list of platform tested by the CI.

04123ab CI: Skip documentation deployment step outside of forks
097d01d CI: Merge Linux and macOS into a posix runner
9a4d1f2 CI: Run GitHub workflows only on main branch
b59a345 CI: Fix windows pipeline
cdd5e3a CI: Run the tests on Windows 10 and Windows 11 both in Debug and Release
f8067a8 CI: Set GitHub workflow badges to the correct URL
fd1dc37 CI: Skip documentation deployment step on forks

Socket library has been cleaned up, after some "encouragement" given in a issue that was trying to expand UDP support.

645dae9 Socket: Move examples from header to snippets
9755163 Socket: Move SocketNetworking::resolveDNS into its own SocketDNS class
befe282 Socket: Require a valid socket for both SocketClient::connect overloads
cf3034e Socket: Split SocketServer::bind out of SocketServer::listen to allow creating UDP servers

First contributor of the project has appeared!
Thanks to the contributor now both Tools and Plugin will work even if visual studio is installed in a directory different from default.

89a3518 Plugin: Detect Visual Studio path dynamically
978cc4c Tools: Update 7-zip to 24.05

Some issues have been reported by users mixing Sane C++ Libraries with C++ Standard Library and they've been fixed as well.
98e7310 Foundation: Just include <memory.h> if SC_COMPILER_ENABLE_STD_CPP is defined or exceptions are used

And then just a usual bunch of fixes and minor additions.

036c5c4 Documentation: Update README with latest video
0f1cd95 Documentation: Improve commit message format and squashing sections
940b780 Everywhere: Support Visual Studio 2019
f37c814 Everywhere: Fix ClangCL build
1d0271c FileSystem: Fix Path::relativeFromTo
fde2d97 Foundation: Allow passing arbitrary number of arguments to placement new
f86ce3d Strings: Add String::owns
75819d5 Time: Add operator < and > to Milliseconds and HighResolutionCounter::getRelative
40341b4 Tools: Escape path for the .touched file in the NMAKE Windows bootstrap
d30163d Tools: Use just commit hash for SC-Package when cloning git repos
d47e439 Tools: Fix HEAD casing for "git rev-parse" used by SC-Package
d4ce9c3 Tools: Automatically retry building a tool on make failure

What's Changed

Tools & Plugin: Dynamic Visual Studio path detection by @silent-tech in #18

New Contributors

@silent-tech made their first contribution in #18

Full Changelog: release/2024/04...release/2024/05

Releases: Pagghiu/SaneCppLibraries

2025-04

Process

Foundation

Async

Tools

2025-03

Process

Foundation

Async

Tools

Build

2025-02

Containers

SC::Foundation

SC::Build

Minor Changes

2025-01

SC::Async

SC::AsyncStreams

SC::Foundation

SC::Time

Minor changes

2024-12

SC::AsyncStreams

SC::Async

SC::Foundation

Refactoring

Minor changes

2024-11

Async Streams

Others

2024-08

Http

SCExample

Async

Features

Bugs (fixed)

Quality of Life improvements

Others

2024-07

SCExample

Serialization

Plugin

Build

Contributions

Others

What's Changed

New Contributors

Contributors

2024-06

SCExample

SC::Async

SC::Plugin

SC::FileSystemWatcher

SC::Build

Others

2024-05

What's Changed

New Contributors

Contributors