Compressing assets

The DirectX Tool Kit is designed for ease-of-use and to fit neatly into the 'Venn diagram' of all Microsoft platforms. Therefore, the library does not include an asset management system, file-system-in-a-file WAD-style packaging, etc. The Universal Windows Platform (UWP) and Microsoft GDK both make use of MSIX packaging. On Windows 10 and Xbox Series X|S technologies like DirectStorage can greatly improve on-disk compression and throughput. Generally these concerns are out-of-scope for the tool kit.

That said, for very large models SDKMESH and even block-compressed DDS files can have a very large disk footprint. For SDKMESH, some space savings can come from using tricks like the Samples Content Exporter -compressvertexdata:+ option which uses more compact DXGI formats for vertex data, and this also translates into in-memory savings as well. This still only gets you so far. The good news is that SDKMESH and DDS files compress well using classic lossless compression schemes.

If your system requirements are Windows 8 or later, then you can make use of the Compression API as a quick & dirty solution. For some example tooling see:

• xbcompress command-line tool which generates a SZDD/KWAJ-style compressed file using the Compression API.
• ReadCompressedData .cpp / .h which loads these compressed binary blob files and expands them.

Example

Compressing a bin file:

xbcompress mylargedata.bin

Loading a 'compressed' bin file:

auto blob = DX::ReadCompressedData(L"mylargedata.bi_");

Note that the Compression API is optimized for compression ratio and decompression performance, not compression time.

DirectX 11

Examples

Loading a 'compressed' SDKMESH file:

{
    auto modelBlob = DX::ReadCompressedData(L"AbstractCathedral.sdkmes_");
    m_model = Model::CreateFromSDKMESH(device, modelBlob.data(), modelBlob.size(), fxFactory);
}

Loading a 'compressed' DDS file:

{
    auto ddsBlob = DX::ReadCompressedData(L"Map3DColor.dd_");
    DX::ThrowIfFailed(
        CreateDDSTextureFromMemory(device,
            ddsBlob.data(), ddsBlob.size(),
            nullptr, m_map.ReleaseAndGetAddressOf()));
}

DirectX 12

• CompressedTextureFactory which is a simple extension of EffectTextureFactory for DirectX 12 which looks for compressed DDS files (which have the DD_ extension) as part of the model loading process.

Examples

Loading a 'compressed' SDKMESH file:

{
    auto modelBlob = DX::ReadCompressedData(L"AbstractCathedral.sdkmes_");
    m_model = Model::CreateFromSDKMESH(device, modelBlob.data(), modelBlob.size());
}

Loading a 'compressed' DDS file:

{
    auto ddsBlob = DX::ReadCompressedData(L"Map3DColor.dd_");
    DX::ThrowIfFailed(
        CreateDDSTextureFromMemory(device, resourceUpload,
            ddsBlob.data(), ddsBlob.size(), m_map.ReleaseAndGetAddressOf()));
}

Remarks

For Windows 7 compatibility or just for personal preference, you can use open source solutions like zlib, zopfli, etc. or other commercial middleware.

Credits

The SZDD/KWAJ file format has a long history, and dates back to the days of MS-DOS. The compress command would take a file like "myfile.exe" and compress it into "myfile.ex_". The expand tool did the opposite.

"SZDD" (1990) is the signature of the original MS-DOS COMPRESS.EXE / EXPAND.EXE file format. "KWAJ" (1993) is a similar format which supported additional compression methods.

The xbcompress tool is inspired by this simple design.

File offset	Field length	Description
0	8	Magic byte sequence to uniquely identify file format. The magic sequence is 0x41, 0x46, 0x43, 0x57, 0x47, 0x50, 0x53, 0x4d ('AFCWGPSM').
9	1	Compression mode. COMPRESS_ALGORITHM_LZMS (5) or COMPRESS_ALGORITHM_MSZIP (2)
10	1	File format version. Currently 0x41 ('A')
11	2	Last character (UTF-16LE) that was changed to _ when the compressed name was determined. This value is 0 if ._ was added instead.
13	4	Size in bytes of the original uncompressed data block. To keep the code simple, this file format only supports up to 4 GB file sizes.

Keeping in the tradition of SZDD and KAWJ which are both the initials of the developers, the magic signature here refers to the four developers who have worked on the ATG Samples V-team over the years.