First announced back at GDC 2023 Unity AI, the details were incredibly vague, with a short video describing AI like functionality that will be added to the Unity game engine. Today we got more details about the Unity AI initiative with the announcement of Unity Muse and Unity Sentis.
Unity Muse is coming in multiple stages, but the initial announcement is Unity Muse Chat:
Using Muse Chat, you can leverage AI-based search across Unity documentation, training resources, and support content to get well-structured, accurate, and up-to-date information from Unity. Muse Chat helps you find relevant information, including working code samples, to speed up development and troubleshoot issues.
This functionality sounds very similar to the CoPilot X documentation feature and should be a powerful new search tool for Unity game developers. This is just the start for Muse however, as upcoming features are also described:
Over the next few months, you’ll see even more features in our Muse platform, including the ability to create textures and sprites or even fully animate a character, all using natural input.
The other part of today’s announcement is Unity Sentis. Unity Sentis is described as:
Unity Sentis is a literal gamechanger. At a technical level, it bridges neural networks with the Unity Runtime, but the possibilities that Sentis unlocks are near endless.
Unity Sentis enables you to embed an AI model in the Unity Runtime for your game or application, enhancing gameplay and other functionality directly on end-user platforms.
Now tell me if that description sounds similar to another Unity project?
The Barracuda package is a lightweight cross-platform neural network inference library for Unity.
Barracuda can run neural networks on both the GPU and CPU. For details, see Supported platforms.
Currently Barracuda is production-ready for use with machine learning (ML) agents and number of other network architectures.
At this point I’m fairly sure Unity Sentis is a rebrand of the Unity Barracuda project. Both Unity Muse and Unity Sentis are taking beta applications.
Key Links
Unity AI Muse and Sentis Announcement
You can learn more about Unity AI, the new Unity Muse and Unity Sentis announcements in the video below.