The Cocos runtime engine is a platform built into many apps. We talked about how Cocos Play is using this to help build mini apps for companies like Nestia, MangaToon, and Tencent News. It also has no rigid requirements or performance constraints when developers develop their mini games. Some members of the engine team have begun to challenge the limit of our platform.
Recently, YouYou, an engine development engineer, used Cocos Creator's new 3D engine to create a sea surface effect and an automated vegetation creation effect.
We’re delighted to see that the engine team is still doing some exciting things with their free time and are interested in building additional demos while trying to complete complicated development tasks.
We got a chance to have a quick interview with YouYou and learned a lot about his history and the future of Cocos Creator.
Seven years, two transformations. My unbreakable bond with Cocos
When asked about his first memories at Cocos, he shared one of the first projects he worked on. “Before joining the Cocos Creator team, I worked for the Cocos Studio project in Beijing. Development tools were scarce at that time, and neither Spine nor Dragon Bones editing tools were available,” says Youyou. “Dragon Bones was still a Flash tool, and there were few game engines that could use it. I made the earliest Dragon Bones export tool from Flash to Cocos.”
After leaving Cocos, YouYou was still keen on developing engine tools. At that time, he had the idea of using JS + NodeJS to create an editor. “So I used related technologies to develop a 2D terrain + physics editor,” commented Youyou. “Later, I learned that a team in Xiamen had made a more mature editor called FireBall. Excited about this kind of thinking, I volunteered to join this team. It was not until I moved from Beijing to Xiamen to join FireBall that I realized Cocos purchased it. Cocos had already laid out the direction of using web technology to develop a new game editor.”
“As it happens, two months later, FireBall was wholly acquired by Cocos, and I returned to the company in 2015, starting the development of a new project named Cocos Creator. After leaving the team to build the next best engine, I ironically returned to the arms of the Cocos team. Counting how long I’ve been in the company, I have been with Cocos for seven years.”
From Cocos Studio to Cocos Creator, and then from Creator 2D engine to the Creator 3D engine, many changes have taken place. The biggest change for Youyou was the language change from C++ and C# to JavaScript to TypeScript.
“When I first started working with JavaScript, I was like, wow, cool, this has everything,” YouYou told us with excitement. “What you write with it can be run anywhere, and the code can be run directly without compiling. How impressive!”
But with more and more code and more logic built for Cocos Creator, some low-level errors are more likely to appear. YouYou gave us an example. ”A variable that was initially a number became a String type for unknown reasons. If you performed another addition operation, the result would be completely different. Also, regarding engine API comments, JavaScript's API doc generator cannot detect what type of function parameter has been returned, so it is necessary to manually write each parameter comment when writing an API, which is not only tiring but also prone to errors.”
When it comes to the Cocos Creator 3D engine project, considering the large-scale collaboration needed for the editor, the engine itself, and the community, as well as the needs of a large number of medium and heavy games from external developers, the team decided to rewrite it all in TypeScript. “The stability of the entire editor has improved a lot,” says YouYou. “Using TypeScript to develop allows you to enjoy all the results of JavaScript development, and can help you detect code errors, and generate code hints more easily. It’s so impressive. Sometimes you may even find that a few hundred lines of code can pass the test all in one build.”
Internal adjustment shifts the focus on making adaptable plugins
After transferring to the Cocos Creator project team, Youyou said he is now mainly responsible for the overall rendering framework, the 3D rendering, and the development of various engine modules and editor modules. But he still wants to see how the development will focus on 3D and currently responsible for building these demos.
It is precisely because of the shift of focus that after basically completing the development of Cocos Creator 2.4.x, YouYou plans to make a plugin to get developers familiar with the overall architecture of the Cocos Creator 3D engine and editor more quickly and comprehensively, so that developers can be better integrated into the 3D team.
The ocean effects and automated tree effects shown above are actually his initial development goals. In the beginning, he planned to make a spline curve editing tool. To make the demo scenes more effective, he implemented various special effects.
The goals of this curve tool were:
- Support commonly used curve editing functions;
- Provide Cocos Creator 3.0 with the plugin interface needed to make advanced plugins and other things it lacks;
- Used as an example for developers to make their own advanced plugins
The curve tool requires a powerful Gizmo customization function. Each curve has many control points, and each control point also has multiple control points.
The code of the Gizmo and the Controller can be found in the src/editor directory of spline-tool:
When the curve editing function is relatively complete, Youyou used this curve tool to make a highway. The parts on this highway can be added freely according to the style. For example, if you want to add another railing, you can add a child node and Mesh Tilling component under the spline node and set the corresponding prefab.
The highway is done but still looks a bit bland. To enrich the scene, for example, a highway that passes through a quiet jungle. In this case, terrain and vegetation must be added to the scene.
The current terrain system does not have the function of building vegetation, and Youyou doesn’t want to paint the vegetation bit by bit manually. Therefore, Youyou created a process that automatically brushes on the vegetation on the terrain or objects according to the area, and can support setting the density of vegetation in a specific area. For example, we don’t want to brush vegetation in the highway area.
The figure below is a sample of vegetation production. There are three areas, the hook area is the designated vegetation area, the line area on the left indicates that this area does not produce any vegetation, and the small area on the right indicates that the vegetation density is relatively low. As you can see, the vegetation can not only be generated on the plane, but the sphere above can also be generated with vegetation, and the area will be covered with the plane below.
After combining the highway and vegetation together with after-effect processing, you can see the effect as the following video:
The highway and the vegetation are done. Youyou adds water.
What's interesting here is that the suspension height of the bridge can be controlled by the curve editing of the inspector to control the floating range.
The water is done. But Youyou still want to make it look better, so he started to improve it.
At the beginning, the water effect is built by adding normal map and UV animations to the surface to simulate the surface flow of water. The light reflection effect of the water surface is also automatically calculated using the built-in PBR calculation process.
Next, Youyou adds a wave effect. The method adds a GerstnerWave to the vertex shader.
In the picture above, the boundary line between the water and the wall is straight. After adding the wave effect, the water surface has undulations, and it looks more vibrant.
Then, it's time to achieve the foam floating effect where the water surface contacts the object. This is the most troublesome and difficult part of all the effects built by Youyou.
First, you need to generate the depth information of the object to a depth map, then read the depth map information in the fragment shader of the water surface and compare it with the current water surface depth, and generate foam within a certain depth range where the depth of the object is detected.
The depth map is implemented through a custom rendering pipeline. In the rendering process, you can collect the opaque type model of the camera that needs to generate the depth map, and use the shadowmap pass in the model material to draw into the depth map.
At present, the custom rendering pipeline is still very cumbersome, but the engine team is also actively working to make a better custom rendering pipeline function.
Next, Youyou simulates the effect of shallow water and deep water. The depth information has been generated in the above processing. Here, it is only necessary to mix the shallow and deep color values according to the depth of the water. Finally, add underwater waves based on the depth information, and the effect of the water is complete.
The future is long, but we are all becoming more mature
When asking about his work in Cocos, Youyou smiles. “I am a very early member of the Cocos team. I can say that I grew up with Cocos. Personally, I feel that my own journey for knowledge is very similar to the development of Cocos.”
“Initially, Cocos2d-x still needed the help of the external editor, and then we developed our own editor with Cocos Studio. After that, we upgraded it to Cocos Creator 2D. And now, it has evolved into adding a 3D engine with all dimensions of the world. These are all step by step, bravely trying and building to a higher dimension.”
In the process of technological transformation and direction change, we all ask ourselves whether this direction is right and whether we can afford the responsibility. Youyou is very happy to have withstood the pressure and gained the support and trust of the developers. Cocos is becoming more mature and more respected
Youyou finally remarks before getting back to work, “I am very fortunate that the engine team I belong to has given us full freedom. In the team, basically, everyone can choose the direction they are interested in. If you want to achieve any high-end function, you can also arrange your time to achieve it yourself. This often gives us a sense of accomplishment and more passion for this profession.”
“On the other hand, my feelings of ease might be because I’m in Xiamen and live near the company, so it is easy for me to walk to and from work. The company downstairs has a subway entrance, so the transportation is very convenient. Most of my colleagues in the engine group don’t spend much time commuting.”
After Youyou gets off work, he usually chooses to take a swim. And when he’s in a good mood, he will return to the company to type a few lines of code. Swimming is an excellent exercise for him, and he highly recommends it. Outside of work, he also likes to play video games, watch movies, and travel on foot to relax.
“I’m so happy I work with like-minded people at Cocos. We are a group of people who build a new world with our own hands. When we see our creation work successfully, happiness will fill our body.”
We want to thank Youyou for his time and for sharing his journey with Cocos. We can’t wait to share his work when we release the Cocos Creator 3.0 Tech Demo to the public this October.
