In game development, textures occupy not only a lot of package size but also much of the memory. Although traditional image compression formats (such as JPEG, PNG, etc.) can reduce the resource size, they cannot be directly recognized by the GPU. They still need to be loaded into the memory, decoded by the CPU, and converted into RGB/RGBA formats that the GPU can recognize before being transmitted.
In order to avoid these problems, community Cocos Star Writer Chu Youxiang brought a GPU-specific texture compression introduction and his own scheme so that the texture can be directly recognized and rendered by the GPU. This has the following advantages:
- No CPU decoding is required, which saves CPU operations and reduces power consumption
- The texture is directly transferred to the GPU, which avoids memory usage and improves rendering performance
- An efficient compression algorithm reduces the size of the package.
The principle of compressed textures
The main purpose of traditional image compression is storage and transmission. In order to compress as efficiently as possible, a variable compression ratio is used. Therefore, when decompressing, you need to decompress more pixels to read the position of a specific pixel. It is not suitable for random and fast reading and does not take advantage of the GPU's parallel processing advantages.
The compressed texture uses a fixed compression ratio to divide the texture into multiple pixel blocks, each pixel block contains 2*2 or 4*4 pixels, and then each pixel block is compressed, and the compressed pixel information is stored in In a pixel set, the index position of each pixel block is stored in a block index map.
When reading, first convert the texture coordinates into a block index value, then find the corresponding pixel block in the pixel set, and finally, find the texture color value in this pixel block.
If we use a fixed compression ratio, the GPU can process it in parallel so that it can be decompressed quickly. The compression process of textures is done before the program runs, so it does not care about the encoding speed. Therefore, the compression iterates through all the encodings to find the one with the smallest difference from the original pixels, which is also the reason why texture compression takes longer.
By the way, in normal image formats, PNG is lossless compression, but JPEG is a lossy compression. Compressed textures are lossy compression. You just won't notice the difference on the cell phone in the vast majority of cases.
Commonly used compressed texture formats
The use of compressed textures on mobile phones depends on the support of OpenGL ES. OpenGL ES 2.0 itself does not define any texture compression format. It only provides glCompressTexImage2D() method to upload compressed textures by application programs. The format of compressed textures is defined and implemented by various GPU vendors.
OpenGL ES 3.0 provides a compressed texture standard so that all platforms can use the same compressed texture, but it will take a long time for all devices on the market to transition to OpenGL ES 3.0. Therefore, it is still necessary to use different compressed texture formats for different platforms and devices.
The following formats are commonly used in mobile games.
ETC1
ETC1 compresses the 4*4 pixel block into a fixed 64-bit encoding (8 bytes). The 4*4 pixel block is 16 pixels, each of which is 4 bytes, which occupies a total of 64 bytes, so the compression ratio is 64/8=8.
However, ETC1 can only store RGB information and does not apply to textures with transparency. To solve this problem, Cocos Creator additionally writes transparency information in the ETC1 file, namely the ETC1+A format, and its compression ratio is 64/16=4. ETC1/ETC1+A requires OpenGL ES 2.0 (corresponding to WebGL 1.0) environment. Currently, almost all Android phones support ETC1, but iOS does not. The length and width of the ETC1/ETC1+A texture can be unequal, but it is required to be a power of 2.
ETC2
ETC2 is an extension of ETC1. The compression ratio is the same, but the compression quality is higher, and it supports transparent channels and can completely store RGBA information.
ETC2 requires OpenGL ES 3.0 (corresponding to WebGL 2.0) environment. There are still many low-end Android phones that are not compatible. iOS supports OpenGL ES 3.0 starting from iPhone5S.
ETC2 is the same as ETC1. The length and width can be different, but it is required to be a power of 2.
PVRTC
PVRTC4+A is commonly used in Cocos Creator. The compression ratio is the same as ETC. It is supported by the full range of iOS devices but not Android. Also, PVR requires the texture length and width to be equal (square) and a power of 2. For example, a 1280*720 PNG image will become 2048*2048 after conversion, significantly increasing memory consumption.
In the actual measurement, it was also found that the quality of the converted picture was not as good as ETC1, with blur and burrs, which was not suitable for games with high image quality requirements.
Use of compressed textures
The use of compressed textures is very simple, just add the required format according to the build platform. For details, please refer to the official documentation of Cocos Creator. This article won't cover this.
https://docs.cocos.com/creator/3.0/manual/en/asset/compress-texture.html
The Cocos Creator editor also provides the option of converting compressed textures, which are divided into Fast, Slow, etc. according to the conversion speed. The slower the speed, the better the picture quality. But no matter which one you choose, it only affects the display effect and conversion time, and the video memory usage is the same.
In general, the video memory occupied is the file size of the compressed texture. For example, if the file size is 1.5M, the video memory occupied by it is also 1.5M.
When setting the compressed texture format, Creator 2.x version still needs to manually set it one by one. If you want to set all or part of the resources at once, remember that some big guys have provided plug-ins. Of course, it is more convenient to write your own script to traverse and modify the corresponding .meta file.
Here is a script I wrote.
The Code
In the 2.x version of Cocos Creator, when setting the compressed texture format, one resource is required to manually set it up. For a project with thousands of texture resources, this operation is obviously unrealistic.
The function of this code is to traverse the Creator resource directory, automatically set the compressed texture format, and can cancel at any time to restore the original settings.
The default configuration in the code is that the Android platform uses ETC1+A, and the iOS platform uses the ETC2 format. The conversion options are all slow, that is, the highest quality and the slowest compression. Can be modified as needed
const fs = require('fs');
const path = require('path');
const _etcSettings = {
"android": {
"formats": [
{
"name": "etc1",
"quality": "slow"
}
]
},
"ios": {
"formats": [
{
"name": "etc2",
"quality": "slow"
}
]
}
};
let sourcePath = process.argv[2];
let isCompress = parseInt(process.argv[3]);
let lookupDir = function(url) {
if (!fs.existsSync(url)) {
return;
}
fs.readdir(url, (err, files) => {
if (err) {
console.error(err);
return;
}
files.forEach((file) => {
let curPath = path.join(url, file);
let stat = fs.statSync(curPath);
if (stat.isDirectory()) {
lookupDir(curPath); // Find directory
} else {
if (file.indexOf('.meta') >= 0) {
fs.readFile(curPath, (err, data) => {
if (err) {
console.error(err);
return;
}
let obj = JSON.parse(data);
if (obj && obj.platformSettings) {
obj.platformSettings = isCompress ? _etcSettings : {}; // Set compressed texture
let wrdata = JSON.stringify(obj, null, 2);
fs.writeFile(curPath, wrdata, (err) => {
if (err) {
console.error(err);
return;
}
});
}
});
}
}
});
});
}
if (!path.isAbsolute(sourcePath)) {
sourcePath = path.join(__dirname, sourcePath)
}
lookupDir(sourcePath);To use this, make sure Node.JS is running. Save the above code as setETC.js file.
Assuming that the Creator project directory is d:/test, and parameter 1 means using compressed textures, and parameter 0 means canceling compressed textures.
Set compressed texture
node setETC.js d:/test/assets 1Restore default settings
node setETC.js d:/test/assets 0After the command is executed, Creator can start the build.
In Summary
In the actual project, the test result is that the single image, automatic atlas, and TexturePack combined images add up to a Creator project with more than two thousand images. When using PNG, the apk package's size is nearly 500M, and the memory occupies 1.3G. After using compressed textures, the package size is reduced to 150M, and the memory usage is reduced to 600M.
- The compressed texture's initial file size is 1-2 times larger than that of PNG, but after zipping or packaging into an .apk or .ipa file. The size is 1/2-1/3 smaller than that of PNG, which has significant advantages in reducing the total package size.
- Starting from OpenGL ES 2.0, GPU textures support non-power-of-two (for example, pictures smaller than 2048 save more memory than 2048 pictures). At present, all mobile phones are already OpenGL ES 2.0 and above.
- For the Android native platform, if you want to be compatible with as many devices as possible and want to take advantage of the memory and package body, the best choice is ETC1.
- For the iOS native platform, the PVR picture quality is not a good choice. If only publishing games for iPhone5S and above devices, ETC2 is better than PVR.
- OpenGL ES 3.0 started to introduce a new ASTC format, which Android and iOS support. The picture quality is better than PVR, and the texture length and width are not required to be equal to the power of 2. ASTC may be a unified format in the future, but many low-end Android devices currently do not support it.
Thanks to Chu Youxiang's personal experience in the field and sharing his personal project on the issue.
