Compressed Texture Formats in Metal

快来打我* 2022-08-18 15:29 246阅读 0赞

# Compressed Texture Formats in Metal #

[May 5, 2015][May 5_ 2015] by [Warren Moore][]

In this article, we will consider several GPU-friendly compressed texture formats. These formats allow us to trade some image quality for substantial improvements in disk usage and performance. In particular, we will look at the ETC2, PVRTC, and ASTC formats.

[![The sample app showcases a variety of compressed texture formats][]][The sample app showcases a variety of compressed texture formats]

The sample app showcases a variety of compressed texture formats

[Download the sample code for this article here.][]

## Why Use Compressed Textures? ##

In previous articles, we created textures by loading an image into memory, drawing it into a bitmap context, and copying the bits into a Metal texture. These textures used uncompressed pixel formats such as `MTLPixelFormatRGBA8Unorm`.

Metal also supports several compressed formats. These formats allow us to copy image data directly into a texture without decompressing it. These formats are designed to be decompressed on-demand by the GPU, when the texture is sampled rather than when it is loaded.

The formats we will consider below are lossy, meaning they do not perfectly preserve the image contents, as a lossless format like PNG does. Instead, they trade a reduction in image quality for much smaller memory usage. Using compressed texture data therefore greatly reduces the memory bandwidth consumed when sampling textures, and is also more cache-friendly.

## A Brief Overview of Compressed Texture Formats ##

### S3TC ###

S3TC, also known as DXT, was the first compressed format to gain broad adoption, due to its inclusion in DirectX 6.0 and OpenGL 1.3 in 1998 and 2001, respectively. Although S3TC is broadly supported on desktop GPUs, it is not available on iOS devices.

### PVRTC ###

The PVRTC image format was introduced by Imagination Technologies, creators of the PowerVR series of GPUs at the heart of every iOS device. It was first described in a [2003 paper by Simon Fenney][].

PVRTC operates by downsampling the source image into two smaller images, which are upscaled and blended to reconstruct an approximation of the original. It considers blocks of 4×4 or 4×8 pixels at a time, which are packed into one 64-bit quantity. Thus, each pixel occupies 4 bits or 2 bits, respectively.

One significant limitation of using PVRTC format on iOS is that textures must be square, and each dimension must be a power of two. Fortunately, game textures are most commonly produced in dimensions that are compatible with this limitation.

### ETC ###

Ericsson Texture Compression (ETC) debuted in 2005. Similarly to PVRTC’s 4-bit-per-pixel mode, it compresses each 4×4 block of pixels into a single 64-bit quantity, but lacks support for an alpha channel. A subsequent version, ETC2, adds support for 1-bit and 8-bit alpha channels. ETC2 is supported by all Metal-compatible hardware, but PVRTC often offers better quality at comparable file sizes.

### ASTC ###

Advanced Scalable Texture Compression (ASTC) is the most recent compressed texture format supported by Metal. Developed by AMD and supported fairly widely on OpenGL ES 3.0-class hardware, this format incorporates a selectable block size (from 4×4 to 12×12) that determines the compression ratio of the image. This unprecedented flexibility makes ASTC a very attractive choice, but it requires an A8 processor at a minimum, making it unusable on devices such as the iPhone 5s.

## Container Formats ##

The compression format used by a texture is only half of the picture. In order to load a texture from disk, we need to know its dimensions (width and height) and its pixel format. This metadata is often written into the file itself, in the form of a header that precedes the image data. The layout of the header is described by the container format of the image. In this article, we will use two container formats: PVR and ASTC.

### The PVR Container Format ###

Not to be confused with the PVRTC compression format, the PVR container format describes how the data in a texture file should be interpreted. A PVR file can contain uncompressed data or compressed data in numerous formats (including S3TC, ETC, and PVRTC).

There are a few different, mutually-incompatible versions of the PVR format. This means that each version of PVR requires different code to parse correctly. For example, the header structure for the “legacy” format (PVRv2) looks like this:

struct PVRv2Header
    {
        uint32_t headerLength;
        uint32_t height;
        uint32_t width;
        uint32_t mipmapCount;
        uint32_t flags;
        uint32_t dataLength;
        uint32_t bitsPerPixel;
        uint32_t redBitmask;
        uint32_t greenBitmask;
        uint32_t blueBitmask;
        uint32_t alphaBitmask;
        uint32_t pvrTag;
        uint32_t surfaceCount;
    };

The header structure for the most recent version (PVRv3) looks like this:

struct PVRv3Header {
        uint32_t version;
        uint32_t flags;
        uint64_t pixelFormat;
        uint32_t colorSpace;
        uint32_t channelType;
        uint32_t height;
        uint32_t width;
        uint32_t depth;
        uint32_t surfaceCount;
        uint32_t faceCount;
        uint32_t mipmapCount;
        uint32_t metadataLength;
    };

There are more similarities than differences between the headers. The various fields describe the contained data, including the texture’s dimensions and whether the file contains a set of mipmaps or a single image. If the file contains mipmaps, they are simply concatenated together with no padding. The program reading the file is responsible for calculating the expected length of each image.

Note that the header also has `surfaceCount` and (in the case of PVRv3) `faceCount` fields, which can be used when writing texture arrays or cube maps into a single file. We will not use these fields in this article.

### The ASTC Container Format ###

The ASTC container format is custom-tailored to the ASTC compression format. The header indicates the compression block size, which allows you to select the correct pixel format when loading the data into texture memory and calculate the length of the texture data. Here is the layout of the ASTC header:

struct ASTCHeader {
        uint32_t magic;
        unsigned char blockDimX;
        unsigned char blockDimY;
        unsigned char blockDimZ;
        unsigned char xSize[3];
        unsigned char ySize[3];
        unsigned char zSize[3];
    };

The `xSize`, `ySize`, and `zSize` fields are 24-bit quantities, each encoded as three bytes. The compressed image data immediately follows the header.

## Creating Compressed Textures ##

There are various tools for creating compressed textures, both on the command line and with a GUI. Below, we’ll look at two tools, which allow us to create textures in the formats discussed above.

### Creating Textures with texturetool ###

Apple includes a command-line utility with Xcode called `texturetool` that can convert images into compressed texture formats such as PVRTC and ASTC. At the time of this writing, it does not support the PVRv3 container format. Rather, it writes PVR files using the legacy (PVRv2) format, described above. With the `-e` flag, you specify your desired container format: PVR, ASTC, or RAW (no header). With the `-f` format you specify the compression type: PVRTC or ASTC. The `-m` flag signifies that all mipmaps should be generated and written sequentially into the file.

Each compression format has its own flags for selecting the degree of compression. For example, to create a 4 bit-per-pixel PVRTC texture with a PVR legacy header, including all mipmap levels, use the following invocation of `texturetool`:

`texturetool -m -e PVRTC -f PVR --bits-per-pixel-4 -o output.pvr input.png`

To create an ASTC-compressed texture in an ASTC container with an 8×8 block size using the “thorough” compression mode:

`texturetool -e ASTC -f ASTC --block-width-8 --block-height-8 --compression-mode-thorough -o output.astc input.png`

The `-m` flag for generating mipmaps is ignored when writing a texture with an ASTC container.

### Creating Textures with PVRTexToolGUI ###

Imagination Technologies includes a tool in their [PowerVR SDK][] that allows creation of compressed textures with a GUI. `PVRTexToolGUI` allows you to import a PNG and select a variety of compression formats, including ASTC, PVRTC, and ETC2. Unlike `texturetool`, this application uses the PVRv3 container format by default, so you should use code that expects the appropriate header format if you compress textures with this tool.

[![The PVRTexToolGUI from Imagination Technologies allows the user to compress images in a variety of formats][]][The PVRTexToolGUI from Imagination Technologies allows the user to compress images in a variety of formats]

The PVRTexToolGUI from Imagination Technologies allows the user to compress images in a variety of formats

`PVRTexToolGUI` has a command-line counterpart, `PVRTexToolCLI`, that exposes all of the parameters available in the GUI and allows you to write scripts for rapidly batch-converting textures.

## Loading Compressed Texture Data into Metal ##

Loading compressed textures is a two-fold process. First, we read the header of the container format. Then, we read the image data described by the header and hand it to Metal. Regardless of the compression algorithm used, we do not need to do any parsing of the image data ourselves, since Metal is able to understand and decode compressed textures in hardware.

Creating a texture with compressed data is exactly the same as with uncompressed data. First, we create a texture descriptor with the appropriate pixel format and dimensions. If the texture has mipmaps, we indicate so with the `mipmapped` parameter. For example, here is how we create a texture descriptor for a PVRTC texture with 4 bits per pixel for which we have mipmaps:

MTLTextureDescriptor *descriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatPVRTC_RGBA_4BPP
                                                                                         width:width
                                                                                        height:height
                                                                                     mipmapped:YES];

We can then ask a device to produce a texture that matches this descriptor:

id<MTLTexture> texture = [device newTextureWithDescriptor:descriptor];

For each mipmap level in the image, we need to make a separate call to `-replaceRegion:mipmapLevel:withBytes:bytesPerRow:`, as follows:

MTLRegion region = MTLRegionMake2D(0, 0, levelWidth, levelHeight);
    [texture replaceRegion:region mipmapLevel:level withBytes:levelData bytesPerRow:levelBytesPerRow];

Where `levelWidth` and `levelHeight` are the dimensions of the current mip level, `level` is the index of the current mip level (the base level is index 0), and `levelData` is a pointer to the mip level’s image data. `levelBytesPerRow` can be computed as the length of the image data divided by the level’s height.

## The Sample App ##

The sample app includes a demonstration of all of the texture formats mentioned in this article. Tapping the “Switch Textures” button brings up a menu that allows the user to select among the various textures. The menu options for the ASTC options will not be available if the device on which the app is running does not support ASTC compression.

[![The sample app allows you to select among several compression formats supported by Metal][]][The sample app allows you to select among several compression formats supported by Metal]

The sample app allows you to select among several compression formats supported by Metal

[Download the sample code for this article here.][]

## Conclusion ##

In this article, we have discussed the merits of compressed textures and discussed a few of the available formats. We looked at how to create and load textures in the ETC2, PVRTC, and ASTC formats. Generally speaking, ASTC offers the best quality-to-size ratio on hardware where it is available. The PVRTC format should generally be preferred when targeting devices with the A7 processor.

### Acknowledgements ###

Conversations with [Shannon Potter][] provoked my interest in this topic and encouraged me to write this article.

[May 5_ 2015]: http://metalbyexample.com/compressed-textures/
[Warren Moore]: http://metalbyexample.com/author/warrenm/
[The sample app showcases a variety of compressed texture formats]: /images/20220731/585120cc50fb4d7f85ca9fce88c37116.png
[Download the sample code for this article here.]: http://d2jaiao3zdxbzm.cloudfront.net/wp-content/uploads/CompressedTextures.zip
[2003 paper by Simon Fenney]: http://fileadmin.cs.lth.se/cs/Education/EDA075/notes/fenney03texcomp.pdf
[PowerVR SDK]: http://community.imgtec.com/developers/powervr/installers/
[The PVRTexToolGUI from Imagination Technologies allows the user to compress images in a variety of formats]: /images/20220731/e436021182e94c80a2d7cefd345e05a8.png
[The sample app allows you to select among several compression formats supported by Metal]: /images/20220731/f5b61750aefe47f080932119de6cc0da.png
[Shannon Potter]: https://twitter.com/lucastizma