At the Huawei Developer Conference 2022 (HDC2022) last week, Cocos, a strategic partner of Huawei, introduced the latest cooperation and progress between Cocos and Huawei in several areas to developers. Among them, Cocos XR Technical Director Wang Qingli brought the keynote speech "Cocos Empowers Huawei XR Content Creation Ecology." Cocos will continue working with Huawei to expand AR/VR technology applications and promote the ecological prosperity of the industry.
The following is the transcript of the speech.
Hi everyone, I'm Wang Qingli, Technical Director of Cocos XR. Today I'm going to introduce the progress of Cocos and Huawei's cooperation in XR. Through this presentation, you can learn about the current status of Cocos XR development, what development services Cocos provides for Huawei devices, what types of applications can be released, and some of Cocos' co-built solutions with industry eco-partners.
Cocos XR Authoring Tools
Since its establishment in 2010, Cocos has been providing developers and customers with convenient creation tools and reliable solutions based on Cocos' open-source engine capabilities. Cocos' industry applications have expanded from games to automotive, education, XR, virtual characters, metaverse, IoT, etc., helping various industries turn their ideas into reality more efficiently.
The XR development feature extends to Cocos Creator's cross-platform, lightweight, high-performance advantages. The first version, Cocos CreatorXR, went live for developers in September this year. Build XR games, or convert an existing 3D game into an XR game with one click.
Cocos CreatorXR provides many functional components with grabbing, moving, roaming, teleporting, interacting, virtual splitting, face pinching, socializing, calling, online exhibiting, conferencing, and other metaverse-related functions. Developers can achieve functional development simply by setting parameters, dragging and dropping to place prefabricated bodies and logical components, and building and packaging them to the corresponding devices with one click.
This version mainly packages several modules, including device mapping, interaction, virtual mobility, XR UI, etc., focusing on VR domain development. Still, of course, it will also be carried over to the AR domain. In addition, Cocos already has basic development capabilities for AR and WebXR, as well as compatibility with the Cocos Runtime platform, and these features will be released shortly. In the future, we will also focus on low-code development and building in niche areas and gradually improve XR functional modules.
We know that the XR ecosystem is fragmented, but more and more manufacturers have recognized OpenXR as the industry standard over the past few years. This includes XR units such as the Glass, Pico Neo3, Monado, and more devices being added every year.
In general, the Cocos XR authoring tool has the following four main features and advantages: support for rapid conversion of stock projects; multi-platform adaptation, a single development can be quickly packaged and released to multiple devices; provides web previews to improve debugging efficiency; intelligent selection of the optimal rendering backend on the device side.
Helping Huawei XR technology get off the ground
In recent years, Huawei has continued to build the XR ecosystem and already has several XR products and underlying functional SDKs. Cocos enables the development capabilities of Huawei's VR Engine by integrating OpenXR, including device control and signal processing components, various components and model prefabs for joystick controllers, event processing with interaction via interactors, and motion-related movement, steering, and teleportation, etc. We have packaged all of these common logic components so developers can use them directly.
The image above is the result of converting an original Cocos 3D game, Ghost Shooter, to an XR version via Cocos CreatorXR and running it on Huawei VR Glass. In general, simple games can be packaged directly after conversion. Still, some interaction adjustments and UI modifications may be needed for more complex games. With Cocos, developers can efficiently convert stock projects and quickly enrich the XR ecosystem.
In terms of AR, Cocos and Huawei have also started deep cooperation. On the handheld side, the Cocos ARModule module has implemented the underlying interface to Huawei AR Engine and the corresponding editor functions. It has integrated most of the features of AR Engine to support the development of native applications. In addition, we have also incorporated several major AR SDKs.
The above shows the native AR game "Fingertip War" based on Cocos development. You can see that its plane detection, environment scanning, and mesh reconstruction are all relatively accurate. The collision detection between real and virtual objects, physics laws, etc., are also excellent, which shows that the AR Engine's algorithmic capabilities are very powerful.
Cocos Runtime is a high-performance, point-and-click secure runtime environment widely used on mobile phones. All major mobile phone manufacturers basically use it for their instant games and mini-games.
The above demo shows the AR game demo after combining Cocos Runtime and AR Engine.
In addition, the Huawei browser already supports the WebXR standard. We have adapted it to have a basic WebXR development process. Here is the result of the Fingertip War on Huawei Browser after the development of Cocos CreatorXR packaged into a web project. You can see that its stability and rendering effect are not significantly different from the native side.
Whether on the native side, the instant game platform, or a website, I used Fingertip Wars as a case study. This means that when a developer develops a game, it can be built and published on different devices and platforms and installed and deployed in different forms on Huawei simultaneously. For developers, a single development cycle can create multi-channel publishing for cash, which can undoubtedly significantly improve R&D efficiency and cash conversion rate, and the richness of XR content ecology can grow exponentially or exponentially.
Partnering for XR Eco Development
As a provider of development tools in the ecosystem, Cocos will continue to serve as a bridge between developers, content platforms, and device manufacturers, continuously improving its engine and editor, allowing developers to get their ideas off the ground faster, enhancing the richness of content platforms and promoting device sales, so that the entire ecosystem can quickly form a virtuous cycle.
Cocos will continue improving its development tools in the VR/AR field, optimizing performance, interaction, debugging tools, etc. In VR, Cocos will continue to integrate more SDK features, and optimize compatibility and performance consistency across platforms, so that a project can consistently perform on different AR devices.
For content platforms, Cocos Runtime can be built into apps as a runtime environment for point-and-click XR content, improving conversion rates and production efficiency. When built into the device’s system layer, Cocos Runtime can serve as a lightweight, integrated renderer or launcher for XR devices, quickly implementing 3D objects with low-power runtime rendering, enhancing user experience and promoting ecological accumulation.
Cocos has been working extensively and deeply with leading companies in many industries and fields at home and abroad for more than a decade, and there are innovative new needs in the emerging field of XR. Cocos will open up new market space for partners in the XR field and provide XR value-added features and services.
Cocos will also continue to work with Huawei. We look forward to working with more industry partners and developers to explore the frontiers of XR technology and promote the healthy development of the industry ecosystem.
Cocos CreatorXR Plugin Download Link
https://store.cocos.com/app/en/detail/4117
Forum Post (Chinese)
https://forum.cocos.org/t/topic/140479
We will continue to promote deep cooperation and strategic mutual assistance with device manufacturers and content platforms. On the tool's side, we already support most of the mainstream devices in the market. We are pushing forward to cooperate with more manufacturers. On the system side, Cocos will provide some safe, lightweight, and low-power rendering frameworks - which has always been Cocos' strength.
