© Marcel Braghetto 2022.
All rights reserved.

crust / Part 5 - Init SDL

Marcel Braghetto 7 February 2022

Rusty boat: attribution https://www.pexels.com/photo/a-rusty-boat-on-the-seashore-9816335/

In this article we are going to write some of the core parts of crust-main which gets us to the point of being able to initialise the SDL system. We will focus our energy on Windows and MacOS, leaving the other target platforms for later once our main application code has matured, though we will sprinkle a tiny bit of forward thinking as we go to make later parts of the series easier.

We will also need to grow our crust-build application to download non Rust related dependencies and process them into our target platform build code.

SDL2 in Rust

For those who might have read my series A Simple Triangle a few years ago, you may recall that we used the SDL2 library to handle the foundation level interaction with windowing systems, input events and so forth.

The reason for choosing SDL2 is that it is a cross platform library, allowing us to cross compile and run graphics and media programs on all the target platforms we want to support. SDL2 is a C based library but once you get familiar with its (sometimes archaic) APIs it isn’t too scary to use.

In A Simple Triangle we were writing all our code in C++, so it was trivial to call SDL2 APIs via its header files. This time we are using Rust which can’t automatically consume C/C++ header files, though we can actually call C/C++ code by using extern mappings.

I didn’t want to spend a lot of my time trying to map all the SDL2 C functions into Rust code, so instead we will use some crate dependencies to provide these mappings to us. You might remember we already put the SDL2 dependencies into our crust-main/Cargo.toml file:

[dependencies]
sdl2-sys = "0.34.4"

[dependencies.sdl2]
version = "0.34.4"
default-features = false
features = ["use_mac_framework", "image"]

The kind folk who maintain these crates have done the grunt work of mapping the SDL2 C APIs into Rust using interop features such as extern. This is great news for us, but we need to understand that the SDL2 crates are only an API mapping - under the hood the implementation of SDL2 still lives inside C library artifacts that we need to manually include ourselves.

Note: The Emscripten target platform is an exception as it has its own ported version of SDL2 included in its toolchain.

Think of the Rust crates as a veneer or facade over the top of the foreign SDL2 C methods. This is a fairly common thing to do in Rust when you need to call C/C++ code and we will actually be doing some of this ourselves during the project.

Needing to have the actual SDL2 C implementation artifacts bundled alongside our compiled application requires us to manually download, link and include them into our project. If we don’t do this we will face linker errors when trying to compile code that uses SDL2 and runtime errors where our Rust code is trying to call foreign C methods that don’t exist in the running process.

The Rust toolchain will not help us with the fetching or packaging of non crate based dependencies, but that’s one of the main reasons we are writing crust-build - to help bridge those kinds of gaps in our build pipeline!

Core code

In crust-main we are going to need some core utilities and services, similar to what we did for crust-build. There are a few things that we will make an exact duplicate of - there are likely ways to share Rust code between our crust-build and crust-main Rust projects but to keep things simple we will live with a bit of duplication and keep the projects insulated from each other.

You should end up with a structure like this:

Launcher

The first thing we need to do in our application is attempt to initialise the main SDL2 and SDL2 Image libraries. We have already included the Rust SDL2 crate dependency in our project which as mentioned earlier gives us a Rust wrapper over the C API of SDL2, allowing us to call into SDL2 through our Rust code.

We will do the SDL2 initialisation via lib.rs, but write the actual code for it in a separate Rust file named launcher.rs. Create core/launcher.rs and register it:

use crate::{
    core::{failable_unit::FailableUnit, logs},
    log_tag,
};

pub fn launch() -> FailableUnit {
    logs::out(log_tag!(), "Init SDL2 ...");
    sdl2::init()?;

    logs::out(log_tag!(), "Init SDL2 Image ...");
    sdl2::image::init(sdl2::image::InitFlag::PNG)?;

    logs::out(log_tag!(), "SDL2 ready ...");

    Ok(())
}

The launch method is pretty basic - we are asking the sdl2 library to initialise itself, then asking the sdl2::image library to do the same, with support for PNG image files. The last statement simply returns a successful result of type Unit to the caller, however you will notice that both of the SDL2 commands have the ? operator - meaning that they can potentially fail. If either command fails, the launch method will terminate and return a failure result instead.

Note: I’ll only be using PNG image assets but if you want to include support for other image file types such as JPG you can tweak the image initialisation code to suit.

To use this new launch code, edit lib.rs and replace it with:

pub mod core;
pub mod log_tag;

use crate::core::logs;

pub fn main() {
    std::process::exit(match core::launcher::launch() {
        Ok(_) => 0,
        Err(err) => {
            logs::out(log_tag!(), &format!("Fatal error: {:?}", err));
            1
        }
    });
}

Our main method is still intact but we are now performing a match expression on the result of calling to our core::launcher::launch() method - if the result was any kind of error, we will print out a message and exit with a result code of 1 which is the standard result code when something goes wrong, otherwise we return 0 which indicates our program completed successfully.

You may wonder why we put the launch code into the separate launcher.rs source file - it isn’t apparent just yet, but later on when we implement the Emscripten target we will actually need a completely different implementation for our launch code. By extracting it into a file, we can later on add the Emscripten version in its own file. Don’t worry too much about that yet though - we’ll get to that later.

The first build failure

Ok cool - let’s see now what happens when we run our builder over the new code. We will start by fixing the Windows target then loop back and fix the MacOS Console target after. Even if you are doing this series on MacOS, you should implement the Windows code as it will include additional shared functionality in our crust-build project which will be needed for MacOS targets too.

Note: For the rest of this article if I’m not explicit, assume any code changes are applied to the crust-build project rather than crust-main.

Running our debugger for the Windows target produces output like this (I have abbreviated a lot of the toolchain noise):

[ compile ] Compiling application ...

<snip>cargo rustc  --manifest-path "<snip>\\crust-main\\Cargo.toml" --bin crust --target-dir "<snip>\\windows\\.rust-build\\rust"
   Compiling crust v1.0.0 (<snip>\crust-main)
error: linking with `link.exe` failed: exit code: 1181
  <snip>
  = note: LINK : fatal error LNK1181: cannot open input file 'SDL2.lib'

Oh dear, we can’t even compile our Windows target - but there is a significant clue here about why:

LINK : fatal error LNK1181: cannot open input file 'SDL2.lib'

In Windows if we need to compile against a dynamic library such as SDL2 we need to have two things:

Fixing the SDL2 library dependency

The SDL2 library can be downloaded from the offical site: https://www.libsdl.org/release. The SDL2 Image library can be found at: https://www.libsdl.org/projects/SDL_image/release.

For the Windows target we will be using the Visual C++ development libraries for Windows, which is why we needed to have Visual Studio installed on Windows to give us the MSVC toolchains (not to be confused with Visual Studio Code!). The Windows SDL2 libraries contain the .lib and .dll files we need.

So, a few things to do here in our crust-build project:

Downloading and unzipping

Downloading the SDL2 libraries will require our Rust code to know how to do network operations. There are numerous networking crates for Rust but we will go with one that seems to be pretty popular named reqwest. Add the following to the dependencies of crust-build/Cargo.toml:

reqwest = { version = "0.11", features = ["blocking"] }

We are specifying that we want the blocking features of the crate as it is totally fine for our builder to block synchronously while it downloads files - in fact it is desirable for our use case.

After downloading our files the next thing we need to do is unzip them, so we’ll use the very popular zip crate to help with this. Add it to our dependencies as well:

zip = "0.5.12"

core/downloads.rs

First up, we will introduce a new Rust service that can download something from a given URL and save it to a file. Create a new file core/downloads.rs and register it:

use crate::{
    core::{failable_unit::FailableUnit, io, logs},
    log_tag,
};
use std::path::Path;

pub fn download(url: &str, destination: &Path) -> FailableUnit {
    logs::out(log_tag!(), &format!("Download: {:?}", url));
    logs::out(log_tag!(), &format!("Into: {:?}", destination));

    let response = reqwest::blocking::get(url)?;

    if !response.status().is_success() {
        return Err("Url request was not successful.".into());
    }

    let content = response.bytes()?;

    io::write_bytes(&content, &destination.to_path_buf())?;
    logs::out(log_tag!(), &format!("Download complete: {:?}", destination));

    Ok(())
}

The download method takes a url to fetch from and a destination of where to save the resulting file. We use the reqwest library to perform a blocking GET request against the url then check if the status code is in the success range - short circuiting if it wasn’t. The get request itself will also short circuit with an error if there is a problem downloading the file.

If we were able to successfully download the file we then use the io::write_bytes method we wrote in an earlier article to save it to the destination.

core/io.rs

The files we will be downloading are .zip files so once we have them we still need a way to unzip them. We included the zip crate but we still need a utility method to orchestrate unzipping a file path into a destination path. Edit core/io.rs and add a new method:

pub fn unzip(source: &PathBuf, destination: &PathBuf) -> FailableUnit {
    logs::out(log_tag!(), &format!("Unzipping: {:?} => {:?}", source, destination));

    create_dir(destination)?;

    let zip_file = File::open(&source)?;
    let mut archive = zip::ZipArchive::new(zip_file)?;

    for i in 0..archive.len() {
        let mut file = archive.by_index(i)?;
        let outpath = match file.enclosed_name() {
            Some(path) => destination.join(path.to_owned()),
            None => continue,
        };

        if (&*file.name()).ends_with('/') {
            create_dir(&outpath)?;
        } else {
            if let Some(parent) = outpath.parent() {
                if !parent.exists() {
                    create_dir(&parent.to_path_buf())?;
                }
            }

            let mut outfile = File::create(&outpath)?;
            std::io::copy(&mut file, &mut outfile)?;
        }

        // If the file has any Unix permissions we want to retain them, on Windows this would be a no-op.
        if let Some(mode) = file.unix_mode() {
            apply_permissions(&outpath, mode)?;
        }
    }

    Ok(())
}

A lot of this code is actually derived from the sample code in the zip crate: https://github.com/zip-rs/zip/blob/master/examples/extract.rs.

We are also going to need the ability to rename files or directories, so add the following to core/io.rs as well:

pub fn rename(source: &PathBuf, destination: &PathBuf) -> FailableUnit {
    logs::out(log_tag!(), &format!("Renaming: {:?} => {:?}", source, destination));

    std::fs::rename(source, destination)?;
    Ok(())
}

core/remote_zips.rs

Now that we have a way to download files and a way to unzip files we can create a service designed to orchestrate these actions together. Create core/remote_zips.rs:

use crate::{
    core::{downloads, failable_unit::FailableUnit, io, logs},
    log_tag,
};
use std::{fs::DirEntry, path::PathBuf};

pub fn fetch(url: &str, destination_dir_name: &str, destination_parent_dir: &PathBuf) -> FailableUnit {
    let target_dir = destination_parent_dir.join(destination_dir_name);

    if target_dir.exists() {
        logs::out(log_tag!(), &format!("Destination already exists, skipping download: {:?}", &target_dir));
        return Ok(());
    }

    io::in_temp_dir(&mut |temp_dir| {
        let download_file_path = temp_dir.to_path_buf().join("download.zip");
        downloads::download(&url, &download_file_path)?;

        let unzipped_dir = temp_dir.join("unzipped");
        io::unzip(&download_file_path, &unzipped_dir)?;

        // We will now massage the name of the unzipped directory to be whatever the caller specified. The directory to rename will be the first child of the 'unzipped' directory where we just unzipped the files.
        let content_dir = unzipped_dir.join(&destination_dir_name);
        io::rename(&std::fs::read_dir(&unzipped_dir)?.filter_map(|e| e.ok()).collect::<Vec<DirEntry>>()[0].path(), &content_dir)?;
        io::create_dir(&destination_parent_dir)?;
        io::copy(&content_dir, &destination_parent_dir)
    })
}

The flow of this service is:

Setup SDL2 (Windows)

Alrighty we should now have what we need to fetch the SDL2 libraries during our Windows build. Let’s take it for a spin - edit windows.rs and add some constants to define the URLs and the names of the destination directories for SDL2 and tweak the use statements with a few things we now need (such as remote_zips, Failable and PathBuf):

use crate::{
    core::{
        context::Context, failable::Failable, failable_unit::FailableUnit, logs, outputs, remote_zips, script::Script, scripts,
    },
    log_tag,
};
use std::path::PathBuf;

const SDL2_URL: &str = "https://www.libsdl.org/release/SDL2-devel-2.0.14-VC.zip";
const SDL2_DIR: &str = "sdl2";

const SDL2_IMAGE_URL: &str = "https://www.libsdl.org/projects/SDL_image/release/SDL2_image-devel-2.0.5-VC.zip";
const SDL2_IMAGE_DIR: &str = "sdl2-image";

Introduce a new method that will fetch the SDL2 library which returns the path where unzipped .lib and .dll files can be found:

fn setup_sdl2(context: &Context) -> Failable<PathBuf> {
    remote_zips::fetch(SDL2_URL, SDL2_DIR, &context.working_dir)?;
    Ok(context.working_dir.join(SDL2_DIR).join("lib").join("x64"))
}

You can see that we are using our new remote_zips service to fetch the SDL2 zip file, specifying the working_dir/sdl2 as the destination. For Windows this would resolve to: windows/.rust-build/sdl2.

The returned path will let us know where to find the .lib and .dll files which we need to know to successfully link to SDL2 in our compilation command - it would resolve to: windows/.rust-build/sdl2/lib/x64.

Note: We will only be building 64 bit applications for Windows.

Add another method for SDL2 Image that does a very similar thing:

fn setup_sdl2_image(context: &Context) -> Failable<PathBuf> {
    remote_zips::fetch(SDL2_IMAGE_URL, SDL2_IMAGE_DIR, &context.working_dir)?;
    Ok(context.working_dir.join(SDL2_IMAGE_DIR).join("lib").join("x64"))
}

build

Update our existing build method to now:

pub fn build(context: &Context) -> FailableUnit {
    context.print_summary();
    let sdl2_libs_dir = setup_sdl2(context)?;
    let sdl2_image_libs_dir = setup_sdl2_image(context)?;
    compile(context, &sdl2_libs_dir, &sdl2_image_libs_dir)?;
    create_output(context, &sdl2_libs_dir, &sdl2_image_libs_dir)
}

compile

Let’s fix up the compile method now to take the new path arguments and use them in the Rust compilation command to successfully link against the .lib files:

fn compile(context: &Context, sdl2_libs_dir: &PathBuf, sdl2_image_libs_dir: &PathBuf) -> FailableUnit {
    logs::out(log_tag!(), "Compiling application ...");

    scripts::run(&Script::new(&format!(
        r#"cargo rustc {} --manifest-path {:?} --bin crust --target-dir {:?} -- -L {:?} -L {:?}"#,
        context.variant.rust_compiler_flag(),
        context.source_dir.join("Cargo.toml"),
        context.rust_build_dir,
        sdl2_libs_dir,
        sdl2_image_libs_dir,
    )))?;

    logs::out(log_tag!(), "Compile completed successfully!");

    Ok(())
}

Now when we compile our Rust code we will add extra linker search paths using the -L flag, so our build can locate the appropriate SDL .lib files to link against. Note that the .lib file doesn’t contain the implementation - that is what the .dll files do and we’ll collect them in the create_output method.

Note: If you wanted to add more external libraries to your project you would need to add a search path to the location of their .lib files too.

create_output

Finally we will update the create_output method, so it also collects the appropriate .dll files when it is pulling together the build outputs along with the compiled .exe file:

fn create_output(context: &Context, sdl2_libs_dir: &PathBuf, sdl2_image_libs_dir: &PathBuf) -> FailableUnit {
    logs::out(log_tag!(), "Creating product ...");
    outputs::clean(context)?;
    outputs::collect(
        context,
        vec![
            context.rust_build_dir.join(context.variant.id()).join("crust.exe"),
            sdl2_libs_dir.join("SDL2.dll"),
            sdl2_image_libs_dir.join("SDL2_image.dll"),
            sdl2_image_libs_dir.join("libpng16-16.dll"),
            sdl2_image_libs_dir.join("zlib1.dll"),
        ],
    )
}

Run our project again, and you should see our build pipeline downloading and unzipping SDL2 libraries and compiling the application:

> Executing task in folder crust-build: cargo run -- --target windows --variant debug <

   Compiling crust-build v1.0.0 (<snip>\crust-build)
    Finished dev [unoptimized + debuginfo] target(s) in 1.87s
     Running `target\debug\crust-build.exe --target windows --variant debug`
[ print_summary ] ---------------------------------------------
[ print_summary ] Assets dir:          "<snip>\\crust-main\\assets"        
[ print_summary ] Working dir:         "<snip>\\windows\\.rust-build"
[ print_summary ] Rust build dir:      "<snip>\\windows\\.rust-build\\rust"
[ print_summary ] Variant:             Debug
[ print_summary ] Target home dir:     "<snip>\\windows"
[ print_summary ] Main source dir:     "<snip>\\crust-main"
[ print_summary ] ---------------------------------------------

[ download ] Download: "https://www.libsdl.org/release/SDL2-devel-2.0.14-VC.zip"
[ download ] Into: "<snip>\\Temp\\crustTfaQqH\\download.zip"
[ download ] Download complete: "<snip>\\Temp\\crustTfaQqH\\download.zip"
[ unzip ] Unzipping: "<snip>\\Temp\\crustTfaQqH\\download.zip" => "<snip>\\Temp\\crustTfaQqH\\unzipped"
[ rename ] Renaming: "<snip>\\Temp\\crustTfaQqH\\unzipped\\SDL2-2.0.14" => "<snip>\\Temp\\crustTfaQqH\\unzipped\\sdl2"

<snip>\Temp\crustqCqfE1>xcopy /E /H /I "<snip>\\Temp\\crustTfaQqH\\unzipped" "<snip>\\windows\\.rust-build"

125 File(s) copied

[ download ] Download: "https://www.libsdl.org/projects/SDL_image/release/SDL2_image-devel-2.0.5-VC.zip"
[ download ] Into: "<snip>\\Temp\\crustlcAEwj\\download.zip"
[ download ] Download complete: "<snip>\\Temp\\crustlcAEwj\\download.zip"
[ unzip ] Unzipping: "<snip>\\Temp\\crustlcAEwj\\download.zip" => "<snip>\\Temp\\crustlcAEwj\\unzipped"
[ rename ] Renaming: "<snip>\\Temp\\crustlcAEwj\\unzipped\\SDL2_image-2.0.5" => "<snip>\\Temp\\crustlcAEwj\\unzipped\\sdl2-image"

<snip>\Temp\crustrIP8m9>xcopy /E /H /I "<snip>\\Temp\\crustlcAEwj\\unzipped" "<snip>\\windows\\.rust-build"

28 File(s) copied

[ compile ] Compiling application ...

<snip>cargo rustc  --manifest-path "<snip>\\crust-main\\Cargo.toml" --bin crust --target-dir "<snip>\\windows\\.rust-build\\rust" -- -L "<snip>\\windows\\.rust-build\\sdl2\\lib\\x64" -L "<snip>\\windows\\.rust-build\\sdl2-image\\lib\\x64"
   Compiling crust v1.0.0 (<snip>\crust-main)
    Finished dev [unoptimized + debuginfo] target(s) in 0.53s
[ compile ] Compile completed successfully!
[ create_output ] Creating product ...
[ delete ] Deleting "<snip>\\windows\\out\\debug"
[ collect ] Collecting: "<snip>\\windows\\.rust-build\\rust\\debug\\crust.exe"
[ collect ] Collecting: "<snip>\\windows\\.rust-build\\sdl2\\lib\\x64\\SDL2.dll"
[ collect ] Collecting: "<snip>\\windows\\.rust-build\\sdl2-image\\lib\\x64\\SDL2_image.dll"
[ collect ] Collecting: "<snip>\\windows\\.rust-build\\sdl2-image\\lib\\x64\\libpng16-16.dll"
[ collect ] Collecting: "<snip>\\windows\\.rust-build\\sdl2-image\\lib\\x64\\zlib1.dll"

When the debug launcher runs our main application it will close almost immediately as we don’t yet have any kind of main loop which runs loops continously until there is a reason to exit. However, if you look in the windows/out/debug directory you will see that along with crust.exe we now also have a bunch of .dll files too:

To prove things are working you can open the output directory in a terminal and run crust.exe - you should see our logging commands about SDL2 successfully display:

PS <snip>\windows\out\debug> .\crust.exe
[ launch ] Init SDL2 ...
[ launch ] Init SDL2 Image ...
[ launch ] SDL2 ready ...

You can also rerun our debugger from Visual Studio Code and see that the SDL2 libraries will only be downloaded the first time:

[ fetch ] Destination already exists, skipping download: "<snip>\\windows\\.rust-build\\sdl2"   
[ fetch ] Destination already exists, skipping download: "<snip>\\windows\\.rust-build\\sdl2-image"
[ compile ] Compiling application ...

Awesome - Windows is now compiling correctly with SDL2!

Note: If you want to try out the release build, open a terminal in the crust-build directory and run cargo run -- --target windows --variant release. The out directory will then contain a release version of crust.

Setup SDL2 (MacOS Console)

If you are on a Mac computer I hope you still followed the implementation for Windows because we’ll be using some of the same services to download and unzip files.

On MacOS we will be using the SDL2 libraries that contain MacOS Frameworks. The concept is not too different than for Windows - we need to both link against the SDL2 libraries at compile time, and be able to load the libraries dynamically at runtime.

While implementing the Windows solution was rather straight forward, we are going to have some extra hurdles to jump to get our project running on MacOS, including:

If we run our debug application on MacOS at the moment, we’ll get a failure with output like this (I’ve trimmed out some of the toolchain output):

> Executing task in folder crust-build: cargo run -- --target macos-console --variant debug <

    Finished dev [unoptimized + debuginfo] target(s) in 0.26s
     Running `target/debug/crust-build --target macos-console --variant debug`
[ print_summary ] ---------------------------------------------
[ print_summary ] Assets dir:          "<snip>/crust-main/assets"
[ print_summary ] Working dir:         "<snip>/macos-console/.rust-build"
[ print_summary ] Rust build dir:      "<snip>/macos-console/.rust-build/rust"
[ print_summary ] Variant:             Debug
[ print_summary ] Target home dir:     "<snip>/macos-console"
[ print_summary ] Main source dir:     "<snip>/crust-main"
[ print_summary ] ---------------------------------------------
[ compile ] Compiling application ...
   Compiling crust v1.0.0 (<snip>/crust-main)
error: linking with `cc` failed: exit status: 1
  = note: ld: framework not found SDL2
          clang: error: linker command failed with exit code 1 (use -v to see invocation)

This looks very similar to the problems we had on Windows - the linker cannot find the required SDL2 artifact to link against. For MacOS this would be the SDL2 framework. We would get the same kind of error for SDL2 Image as well. I guess we need to figure out how to construct the SDL2 frameworks and include them in the build pipeline!

SDL2 Framework

We will construct the SDL2 framework first and do SDL2 Image afterward. The flow of what we need to do is:

The other thing to note is that in a subsequent article we will be implementing the bundled MacOS Desktop application target, which will need the same frameworks as our MacOS Console target. To help with this, we will write the code to build the SDL2 frameworks in a way that can be shared between the two target implementations.

Note: Be sure to have installed Xcode, its command line tools and xcodegen before proceeding, as per the instructions in part 1.

Create a new file macos_sdl.rs as a sibling to macos_console.rs, register it via main.rs with: mod macos_sdl; and put the following into it:

use crate::{
    core::{context::Context, failable::Failable, failable_unit::FailableUnit, io, logs, remote_zips, script::Script, scripts},
    log_tag,
};
use std::path::PathBuf;

const SDL2_URL: &str = "https://www.libsdl.org/release/SDL2-2.0.14.zip";
const SDL2_DIR: &str = "SDL2";
const SDL2_FRAMEWORK_NAME: &str = "SDL2.framework";

pub fn setup(context: &Context) -> Failable<PathBuf> {
    let frameworks_dir = context.working_dir.join("Frameworks");

    io::create_dir(&frameworks_dir)?;
    setup_sdl2(context, &frameworks_dir)?;

    Ok(frameworks_dir)
}

fn setup_sdl2(context: &Context, frameworks_dir: &PathBuf) -> FailableUnit {
    let output_dir = frameworks_dir.join(SDL2_FRAMEWORK_NAME);
    if output_dir.exists() {
        return Ok(());
    }

    remote_zips::fetch(SDL2_URL, SDL2_DIR, &context.working_dir)?;

    let xcode_project_dir = context.working_dir.join(SDL2_DIR).join("Xcode").join("SDL");
    logs::out(log_tag!(), "Compiling Xcode framework for SDL2, this may take a while ...");

    scripts::run(&Script::new(
		r#"xcodebuild archive -scheme Framework -destination "platform=macOS" -archivePath ./SDL2.xcarchive SKIP_INSTALL=NO BUILD_LIBRARY_FOR_DISTRIBUTION=YES"#
	).working_dir(&xcode_project_dir))?;

    io::copy(
        &xcode_project_dir.join("SDL2.xcarchive").join("Products").join("Library").join("Frameworks").join(SDL2_FRAMEWORK_NAME),
        &output_dir,
    )
}

The setup method will create a new Frameworks directory in the working directory, for the MacOS Console target this would resolve to macos-console/.rust-build/Frameworks.

The setup_sdl2 method will see if SDL2.framework already exists and short circuits if it does. Otherwise the SDL2 source code is downloaded and unzipped into macos-console/.rust-build/SDL2 then the xcodebuild command shown is run within the SDL2 Xcode project at macos-console/.rust-build/SDL2/Xcode/SDL.

The xcodebuild command uses the following noteworthy arguments:

Once the Xcode build is complete the SDL2.framework which is buried inside the compiled SDL2.xcarchive directory is copied out into the Frameworks directory we created. We would end up with a structure like this:

+ root

    + macos-console
        + .rust-build
            + Frameworks
                + SDL2.framework

Note: You can learn more about how the SDL2 source code works by manually opening its Xcode project in the SDL2 source code under the Xcode/SDL directory - this is how I figured out how and what to invoke on the command line.

We still have to create the SDL2 Image framework but before that let’s wire in the new code. Edit the build method in macos_console.rs:

pub fn build(context: &Context) -> FailableUnit {
    context.print_summary();
    let frameworks_dir = macos_sdl::setup(context)?;

You will need to add macos_sdl to the use block as well. Now if you run the project you should see the SDL2 source library download, followed by a heap of Xcode build commands. Our overall compilation will still fail until we fix SDL2 Image as well but you should now see the SDL2.framework inside the macos-console/.rust-build/Frameworks directory.

SDL2 Image Framework

The SDL2 Image dependency is a bit tricky because although it comes with an Xcode project in its source files, it also contains a precompiled version of the webp framework which does not contain support for the Apple Silicon (ARM64) architecture. This makes it impossible to compile the SDL2 Image source Xcode project for the arm64 variant. Unfortunately it means we need to either:

I didn’t want to exclude Apple Silicon support because going forward I think it will become almost the default architecture for Mac computers over time, so we will instead get our hands dirty!

What we have to do:

I have done the hard work already and figured out what the custom Xcode project needs to include which we will model in an xcodegen recipe, written in the YAML format. We will use this recipe to dynamically construct the Xcode project at build time via xcodegen, then compile it using xcodebuild to produce our final framework.

Before we do that though, we will need our crust-build application to be able to create symlinks - allowing us to avoid copy pasting source files and directories into projects but instead link them via references. Although the Rust language has some support for creating symlinks it doesn’t seem consistent across platforms and also doesn’t appear to accommodate creating relative links - instead preferring absolute paths. Using absolute paths is a problem in a lot of what we are doing because some relative symlinks actually get embedded into the products themselves - particularly for MacOS. We will roll our own symlinking approach by delegating to the underlying operating system commands via our scripts service.

Edit core/io.rs and add a new method that can create symlink:

pub fn create_symlink(source: &PathBuf, target: &PathBuf, working_dir: &PathBuf) -> FailableUnit {
    delete(target)?;

    logs::out(log_tag!(), &format!("Creating symlink: {:?} <=> {:?}", source, target));

    if cfg!(target_os = "windows") {
        scripts::run(&Script::new(&format!("mklink /D {:?} {:?}", &target, &source)).working_dir(&working_dir))
    } else {
        scripts::run(&Script::new(&format!("ln -s {:?} {:?}", &source, &target)).working_dir(&working_dir))
    }
}

The delete command might seem a bit curious here - the reason is that if a symlink already exists but is broken (the thing it points to isn’t there) then we won’t be able to create a working link unless we get rid of the old one first.

We also need to create symlinks differently on Windows, using the mklink command with the target and source in the order shown, whereas on other platforms we can use ln -s but the source argument comes first.

Note: Our scripts service creates batch files for Windows which allows mklink to work without problems, whereas if we were to use PowerShell on Windows symlinks have additional permission security models applied and its harder to do through a script.

xcodegen recipe

We could put the xcodegen recipe in a .yml file but I’m feeling lazy so we’ll just inline it as a Rust constant instead :) Edit macos_sdl.rs and put the following at the bottom:

Important: Do not change the indentation - YAML is an indentation driven markup language, I can’t say I am a fan of that approach but when in Rome!

const SDL2_IMAGE_CUSTOM_FRAMEWORK_PROJECT_DEFINITION: &str = r#"
name: SDL2_image
options:
    createIntermediateGroups: true
    deploymentTarget:
        macOS: "10.12"

targets:
    SDL2_image:
        type: framework
        platform: macOS
        info:
            path: Generated/Info.plist
            properties:
                CFBundleIdentifier: org.libsdl.SDL2-image
                CFBundleVersion: 2.0.5
                CFBundleShortVersionString: 2.0.5
        entitlements:
            path: Generated/app.entitlements
        sources:
            - Source/SDL_image.h
            - Source/IMG.c
            - Source/IMG_ImageIO.m
            - Source/IMG_bmp.c
            - Source/IMG_gif.c
            - Source/IMG_jpg.c
            - Source/IMG_lbm.c
            - Source/IMG_pcx.c
            - Source/IMG_png.c
            - Source/IMG_pnm.c
            - Source/IMG_svg.c
            - Source/IMG_tga.c
            - Source/IMG_tif.c
            - Source/IMG_webp.c
            - Source/IMG_xcf.c
            - Source/IMG_xpm.c
            - Source/IMG_xv.c
            - Source/IMG_xxx.c
        settings:
            DYLIB_COMPATIBILITY_VERSION: 3.0.0
            DYLIB_CURRENT_VERSION: 3.2.0
            CLANG_ENABLE_OBJC_ARC: NO
            GCC_PREPROCESSOR_DEFINITIONS:
                - "$(inherited)"
                - "LOAD_BMP"
                - "LOAD_JPG"
                - "LOAD_PNG"
            HEADER_SEARCH_PATHS:
                - $(SRCROOT)/../SDL2/include
            LIBRARY_SEARCH_PATHS:
                - $(inherited)
                - $(PROJECT_DIR)
                - $(PROJECT_DIR)/Frameworks
        dependencies:
            - framework: Frameworks/SDL2.framework
              embed: false
            - sdk: ApplicationServices.framework
            - sdk: Foundation.framework
"#;

The main difference in the Xcode project defined above and the original SDL2 Image Xcode project is that there are no source files or frameworks related to webp.

Add the following constants near the top of macos_console.rs:

const SDL2_IMAGE_URL: &str = "https://www.libsdl.org/projects/SDL_image/release/SDL2_image-2.0.5.zip";
const SDL2_IMAGE_DIR: &str = "SDL2_image";
const SDL2_IMAGE_FRAMEWORK_NAME: &str = "SDL2_image.framework";
const SDL2_IMAGE_CUSTOM_FRAMEWORK_DIR: &str = "SDL2_image_custom_framework";

Now add the following method which will orchestrate the SDL2 Image build:

fn setup_sdl2_image(context: &Context, frameworks_dir: &PathBuf) -> FailableUnit {
    let output_dir = frameworks_dir.join(SDL2_IMAGE_FRAMEWORK_NAME);

    if output_dir.exists() {
        return Ok(());
    }

    remote_zips::fetch(SDL2_IMAGE_URL, SDL2_IMAGE_DIR, &context.working_dir)?;

    let custom_framework_dir = context.working_dir.join(SDL2_IMAGE_CUSTOM_FRAMEWORK_DIR);

    io::delete(&custom_framework_dir)?;
    io::create_dir(&custom_framework_dir)?;
    io::create_symlink(&context.working_dir.join(SDL2_IMAGE_DIR), &PathBuf::from("Source"), &custom_framework_dir)?;
    io::create_symlink(frameworks_dir, &PathBuf::from("Frameworks"), &custom_framework_dir)?;
    io::write_string(SDL2_IMAGE_CUSTOM_FRAMEWORK_PROJECT_DEFINITION, &custom_framework_dir.join("project.yml"))?;

    scripts::run(&Script::new("xcodegen generate").working_dir(&custom_framework_dir))?;

    logs::out(log_tag!(), "Compiling custom Xcode framework for SDL2_image, this may take a while ...");

    scripts::run(&Script::new(
		r#"xcodebuild archive -scheme SDL2_image -destination "platform=macOS" -archivePath ./SDL2_image.xcarchive SKIP_INSTALL=NO BUILD_LIBRARY_FOR_DISTRIBUTION=YES"#
	).working_dir(&custom_framework_dir))?;

    logs::out(log_tag!(), "Copying SDL2_image.framework into Frameworks directory ...");

    io::copy(
        &custom_framework_dir
            .join("SDL2_image.xcarchive")
            .join("Products")
            .join("Library")
            .join("Frameworks")
            .join(SDL2_IMAGE_FRAMEWORK_NAME),
        &output_dir,
    )
}

There is a fair bit going on here so I’ll try to describe the steps and you can follow the code:

Now update the setup method to invoke our new setup_sdl2_image after setup_sdl2:

pub fn setup(context: &Context) -> Failable<PathBuf> {
    let frameworks_dir = context.working_dir.join("Frameworks");

    io::create_dir(&frameworks_dir)?;
    setup_sdl2(context, &frameworks_dir)?;
    setup_sdl2_image(context, &frameworks_dir)?;

    Ok(frameworks_dir)
}

If you run our application again, it should work its way through downloading SDL2 Image, creating the custom Xcode framework project from it, then building it to end up with SDL2_image.framework.

Although the overall build will still fail you should now see the two SDL frameworks we need in our working directory:

+ root
    + macos-console
        + .rust-build
            + Frameworks
                - SDL2.framework
                - SDL2_image.framework

Linking frameworks

The reason our build is still failing is because we haven’t yet linked the frameworks into our compilation script but now that we have the frameworks we need, we can solve this problem. Edit macos_console.rs and update the build method to use the path returned by our macos_sdl::setup invocation:

pub fn build(context: &Context) -> FailableUnit {
    context.print_summary();
    let frameworks_dir = macos_sdl::setup(context)?;
    compile(context, &frameworks_dir)?;
    create_output(context, &frameworks_dir)
}

Update the compile method to accept the frameworks_dir argument and use it to specify the linker arguments needed:

fn compile(context: &Context, frameworks_dir: &PathBuf) -> FailableUnit {
    logs::out(log_tag!(), "Compiling application ...");

    scripts::run(&Script::new(&format!(
        r#"cargo rustc {} --manifest-path {:?} --bin crust --target-dir {:?} -- -L framework={:?}"#,
        context.variant.rust_compiler_flag(),
        context.source_dir.join("Cargo.toml"),
        context.rust_build_dir,
        frameworks_dir,
    )))?;

    logs::out(log_tag!(), "Compile completed successfully!");

    Ok(())
}

You will also need to update the use block to include the PathBuf type. Note that our compilation script now has the additional -L framework= argument - this tells the linker to look for any required frameworks in the location specified.

Finally we can update the create_output method so it can either copy or symlink the Frameworks directory into the output directory, then associate an rpath with the compiled application, pointing at the Frameworks directory so the SDL2 frameworks can load dynamically at runtime:

fn create_output(context: &Context, frameworks_dir: &PathBuf) -> FailableUnit {
    let output_dir = outputs::output_dir(context);

    logs::out(log_tag!(), "Creating product ...");

    outputs::clean(context)?;
    outputs::collect(context, vec![context.rust_build_dir.join(context.variant.id()).join("crust")])?;

    match context.variant {
        Variant::Debug => {
            logs::out(log_tag!(), "Debug build - symlinking frameworks ...");
            io::create_symlink(&frameworks_dir, &PathBuf::from("Frameworks"), &output_dir)?;
        }

        Variant::Release => {
            logs::out(log_tag!(), "Release build - copying frameworks ...");
            outputs::collect(context, vec![frameworks_dir.clone()])?;
        }
    }

    scripts::run(&Script::new("install_name_tool -add_rpath @loader_path/Frameworks crust").working_dir(&output_dir))
}

For debug builds we will symlink the Frameworks directory to save time, but for release builds we’ll actually make a copy of it. There probably isn’t a strong use case for a release build of the MacOS Console target but we’ll put it in to be more complete. You will also need to add the core::variant::Variant and core::io types to the use block.

The association of the Frameworks directory is done via the install_name_tool, updating the crust binary to be aware of the Frameworks directory next to it - we did the same thing in A Simple Triangle if you had followed that series.

Note: If you want to try out the release build, open a terminal in the crust-build directory and run cargo run -- --target macos-console --variant release. The out directory will then contain a release version of crust, which will have a full copy of the Frameworks directory instead of a symlink. This would allow you to copy it somewhere else and run it.

Does it work ???

Ok, run the application again and this time all the stars should align and after the application finishes you should see our logging output in the terminal window:

If you look in the macos-console directory you would see something like this:

+ root
    + macos-console
        + out
            + debug
                + .rust-build
                + crust
                + Frameworks

Summary

Well I need a break after writing this article - it was pretty heavy going. The good news though is that we now have a functioning SDL2 based application for both Windows and MacOS. In the next article we will revisit our main application code and create a new OpenGL window and add the main loop so we can actually display something.

The code for this article can be found here.

Continue to Part 6: OpenGL window.

End of part 5

© Marcel Braghetto 2022. All rights reserved.