Realm Database for Unity Games (DevBlog #31)

Network Traders, the game I am currently working on, stores parts of the game data in a central, cloud-hosted database. But some of the user-specific data is private in nature and therefore only stored on the player’s mobile device. Until now, I have just serialized this data to persistent storage. But as I needed to add new user-specific properties, I wondered why I should not use a database on the client as well. After all, they may provide benefits such as atomicity or migrations when the database schema changes. After some searching I found Realm, which integrated nicely into my Unity project, although with some intricacies I would have liked to know beforehand.

Why use a Database?

Working on a game with a client app and a backend server, I have a direct comparison between using a database (server backend) and serialized data (client). The server is written in .NET C# and connected to a MariaDB database. I use Entity Framework Core to easily access my data with LINQ statements, so no hassle with SQL queries. When I need to add or remove tables or columns or move them around, I can do so easily with Fluent Migrator migrations.

On the client side, I did what people usually do, I serialized my data into files using the BinaryFormatter. That is fine as long as your data stays the same over a long time or you only add new classes to be serialized. But, like in the backend, once you start moving data around between versions of your game, you need to start storing version information and convert the data for every new client version. Worse still, you never know what client version the players have out there, so all conversion routines have to stay in your code indefinitely. In other words, you need to implement migrations yourself. That was when I seriously started thinking about using a database for the client app as well.

Alternatives

As it was all about convenience, I started looking for a single-user database providing a fluent interface and migrations. And, of course, it had to integrate well into a Unity project. The first one I came across, and which was also recommended by the folks at Reddit and Mastodon, was SQLite. There is a very good description by Rizwan Asif on how to bring SQLite and Unity together. However, I did not really find a convincing solution on how to add a fluent interface and migrations. At this point, I would also like to mention Room, the built-in Android database. But there currently seems to be no API for Unity. So in the end, I gave Realm a try.

What is Realm?

As Wikipedia puts it, Realm “is an open source object database management system”. The Realm devs advertise it as a mobile database and a replacementfor SQLite and ORMs. While Realm is available under the Apache License 2.0, it is owned by MongoDB Inc. It is not, however, based on MongoDB.

Realm is actually a quite comprehensive bundle offering not only local databases, but also synchronization with a backend service called “Atlas”. In my case, however, it was the convenience it provides for storing data locally on a device that made me give it a try. It covers all the requirements I listed above: seamless integration into C# code, migrations, and on top of it a Unity package is provided.

Integration into your Unity Project

Integration of Realm into your Unity project is straight-forward. The installation instructions using the NPM registry are easy to follow, and you are quickly ready for your first steps. The MongoDB blogs page lists quite a few tutorials and examples to get you started with your first Realm.GetInstance().

In order to store data in your new Realm instance, you need to define an object schema for it. This is a class derived from IRealmObject. In my case, for example, I store information about cities in the Realm database, where the schema definition looks like this:

public partial class CityEntity : IRealmObject
{
  [PrimaryKey]
  public string PlayerId { get; set; }
  public string CityName { get; set; }
  public int GoodID { get; set; }
  public string Notes { get; set; }
}

Each player builds their own city on their mobile device, and sends merchants to other players to trade the goods they produce. Read more about this in my article about the idea behind network traders.

Once your data model is defined, you can write these objects to the database using a transaction inside realm.Write(() => {..}). Specific objects are retrieved either using the Find method, or through queries in LINQ syntax.

Hurdles

Using Realm is not difficult, especially if you have worked with other database integrations before. But as with every new piece of software, there are some hurdles you might stumble over, as did I. So here is a short list for troubleshooting what may be common beginner’s mistakes.

1. Proceed step by step. If you intend to introduce Realm into an existing project, consider doing it in small steps. Realm may force you to make changes to your data model. The more different objects you need to store, the more you need to change. I first tried to rework all of my data storage, but realized that this incurs a high risk of errors. In this case, those may be critical as player data may be lost. So I started with a partial rework, with the rest of the data types following one after the other.
2. String comparisons. Realm reacts strangely regarding string comparisons and method calls in where clauses. The call
   realm.All<Entity>().Where(x => x.idString.Equals(guid.ToString()))
   throws two different kinds of errors. Avoid Equals and toString and use the following code instead:
   realm.All<Entity>().Where(x => x.idString == guidString)
   where guidString is the GUID converted to string. The manual page on filtering and sorting explains much of this, but there is plenty of room for mistakes.
3. Persistent storage on Android. This actually cost me quite some headaches. I had a functioning version running in the Unity editor, and then deployed it on an Android device. As database location, I had selected Application.persistentDataPath which I had been using before, too. But now I got an exception saying that this was a “read-only file system”. After asking for – and getting – advice on the Realm forum it turned out that persistentDataPath historically points to “external” storage, which used to be sdcards. Realm specifically uses the internal data path as default. Read the forum thread for more information.

Summary

Realm seems to be what I have been looking for when it comes to conveniently storing game data on a mobile device. However, I still feel like an early adopter sometimes, although Realm has been around for roundabout ten years. Anyway, I spent quite some time adjusting my data models, so I will stick to it for the time being. Next time I start a new project, and if nothing severe happens, I will probably go for Realm from the start. That will hopefully prevent some of the hassles I have gone through with Network Traders.