Andrew Morgan on Databases

Introduction

In the previous post in this series, we saw how to migrate your Realm data when you upgraded your iOS app with a new schema. But, that only handled the data in your local, standalone Realm database. What if you’re using MongoDB Realm Sync to replicate your local Realm data with other instances of your mobile app and with MongoDB Atlas? That’s what this article will focus on.

We’ll start with the original RChat app. We’ll then extend the iOS app and backend Realm schema to add a new feature that allows chat messages to be tagged as high priority. The next (and perhaps surprisingly more complicated from a Realm perspective) upgrade is to make the author attribute of the existing ChatMessage object non-optional.

You can find all of the code for this post in the RChat repo under these branches:

• Starting point
• Upgrade #1
• Upgrade #2

Prerequisites

Realm Cocoa 10.13.0 or later (for versions of the app that you’re upgrading to)

Catch-Up — The RChat App

RChat is a basic chat app:

• Users can register and log in using their email address and a password.
• Users can create chat rooms and include other users in those rooms.
• Users can post messages to a chat room (optionally including their location and photos).
• All members of a chatroom can see messages sent to the room by themselves or other users.

Upgrade #1: Add a High-Priority Flag to Chat Messages

The first update is to allow a user to tag a message as being high-priority as they post it to the chat room:

That message is then highlighted with bold text and a “hot” icon in the list of chat messages:

Updating the Backend Realm Schema

Adding a new field is an additive change—meaning that you don’t need to restart sync (which would require every deployed instance of the RChat mobile app to recognize the change and start sync from scratch, potentially losing local changes).

We add the new isHighPriority bool to our Realm schema through the Realm UI:

We also make isHighPriority a required (non-optional field).

The resulting schema looks like this:

{
   "bsonType": "object",
   "properties": {
     "_id": {
       "bsonType": "string"
     },
     "author": {
       "bsonType": "string"
     },
     "image": {
       "bsonType": "object",
       "properties": {
         "_id": {
           "bsonType": "string"
         },
         "date": {
           "bsonType": "date"
         },
         "picture": {
           "bsonType": "binData"
         },
         "thumbNail": {
           "bsonType": "binData"
         }
       },
       "required": [
         "_id",
         "date"
       ],
       "title": "Photo"
     },
     "isHighPriority": {
       "bsonType": "bool"
     },
     "location": {
       "bsonType": "array",
       "items": {
         "bsonType": "double"
       }
     },
     "partition": {
       "bsonType": "string"
     },
     "text": {
       "bsonType": "string"
     },
     "timestamp": {
       "bsonType": "date"
     }
   },
   "required": [
     "_id",
     "partition",
     "text",
     "timestamp",
     "isHighPriority"
   ],
   "title": "ChatMessage"
 }

Note that existing versions of our iOS RChat app can continue to work with our updated backend Realm app, even though their local ChatMessage Realm objects don’t include the new field.

Updating the iOS RChat App

While existing versions of the iOS RChat app can continue to work with the updated Realm backend app, they can’t use the new isHighPriority field as it isn’t part of the ChatMessage object.

To add the new feature, we need to update the mobile app after deploying the updated Realm backend application.

The first change is to add the isHighPriority field to the ChatMessage class:

class ChatMessage: Object, ObjectKeyIdentifiable {
   @Persisted(primaryKey: true) var _id = UUID().uuidString
   @Persisted var partition = "" // "conversation=<conversation-id>"
   @Persisted var author: String? // username
   @Persisted var text = ""
   @Persisted var image: Photo?
   @Persisted var location = List<Double>()
   @Persisted var timestamp = Date()
   @Persisted var isHighPriority = false
   ...
}

As seen in the previous post in this series, Realm can automatically update the local realm to include this new attribute and initialize it to false. Unlike with standalone realms, we **don’t*- need to signal to the Realm SDK that we’ve updated the schema by providing a schema version.

The new version of the app will happily exchange messages with instances of the original app on other devices (via our updated backend Realm app).

Upgrade #2: Make author a Non-Optional Chat Message field

When the initial version of RChat was written, the author field of ChatMessage was declared as being optional. We’ve since realized that there are no scenarios where we wouldn’t want the author included in a chat message. To make sure that no existing or future client apps neglect to include the author, we need to update our schema to make author a required field.

Unfortunately, changing a field from optional to required (or vice versa) is a destructive change, and so would break sync for any deployed instances of the RChat app.

Oops!

This means that there’s extra work needed to make the upgrade seamless for the end users. We’ll go through the process now.

Updating the Backend Realm Schema

The change we need to make to the schema is destructive. This means that the new document schema is incompatible with the schema that’s currently being used in our mobile app.

If RChat wasn’t already deployed on the devices of hundreds of millions of users (we can dream!), then we could update the Realm schema for the ChatMessage collection and restart Realm Sync. During development, we can simply remove the original RChat mobile app and then install an updated version on our test devices.

To avoid that trauma for our end users, we leave the ChatMessage collection’s schema as is and create a partner collection. The partner collection (ChatMessageV2) will contain the same data as ChatMessage, except that its schema makes author a required field.

These are the steps we’ll go through to create the partner collection:

• Define a Realm schema for the ChatMessageV2 collection.
• Run an aggregation to copy all of the documents from ChatMessage to ChatMessageV2. If author is missing from a ChatMessage document, then the aggregation will add it.
• Add a trigger to the ChatMessage collection to propagate any changes to ChatMessageV2 (adding author if needed).
• Add a trigger to the ChatMessageV2 collection to propagate any changes to ChatMessage.

Define the Schema for the Partner Collection

From the Realm UI, copy the schema from the ChatMessage collection.

Click the button to create a new schema:

Set the database and collection name before clicking “Add Collection”:

Paste in the schema copied from ChatMessage, add author to the required section, change the title to ChatMessageV2, and the click the “SAVE” button:

This is the resulting schema:

{
   "bsonType": "object",
   "properties": {
     "_id": {
       "bsonType": "string"
     },
     "author": {
       "bsonType": "string"
     },
     "image": {
       "bsonType": "object",
       "properties": {
         "_id": {
           "bsonType": "string"
         },
         "date": {
           "bsonType": "date"
         },
         "picture": {
           "bsonType": "binData"
         },
         "thumbNail": {
           "bsonType": "binData"
         }
       },
       "required": [
         "_id",
         "date"
       ],
       "title": "Photo"
     },
     "isHighPriority": {
       "bsonType": "bool"
     },
     "location": {
       "bsonType": "array",
       "items": {
         "bsonType": "double"
       }
     },
     "partition": {
       "bsonType": "string"
     },
     "text": {
       "bsonType": "string"
     },
     "timestamp": {
       "bsonType": "date"
     }
   },
   "required": [
     "_id",
     "partition",
     "text",
     "timestamp",
     "isHighPriority",
     "author"
   ],
   "title": "ChatMessageV2"
 }

Copy Existing Data to the Partner Collection

We’re going to use an aggregation pipeline to copy and transform the existing data from the original collection (ChatMessage) to the partner collection (ChatMessageV2).

You may want to pause sync just before you run the aggregation, and then unpause it after you enable the trigger on the ChatMessage collection in the next step:

The end users can continue to create new messages while sync is paused, but those messages won’t be published to other users until sync is resumed. By pausing sync, you can ensure that all new messages will make it into the partner collection (and so be visible to users running the new version of the mobile app).

If pausing sync is too much of an inconvenience, then you could create a temporary trigger on the ChatMessage collection that will copy and transform document inserts to the ChatMessageV2 collection (it’s a subset of the ChatMessageProp trigger we’ll define in the next section.).

From the Atlas UI, select “Collections” -> “ChatMessage”, “New Pipeline From Text”:

Paste in this aggregation pipeline and click the “Create New” button:

[
 {
   '$addFields': {
     'author': {
       '$convert': {
         'input': '$author',
         'to': 'string',
         'onError': 'unknown',
         'onNull': 'unknown'
       }
     }
   }
 },
 {
   '$merge': {
     into: "ChatMessageV2",
     on: "_id",
     whenMatched: "replace",
     whenNotMatched: "insert"
   }
 }
]

This aggregation will take each ChatMessage document, set author to “unknown” if it’s not already set, and then add it to the ChatMessageV2 collection.

Click “MERGE DOCUMENTS”:

ChatMessageV2 now contains a (possibly transformed) copy of every document from ChatMessage. But, changes to one collection won’t be propagated to the other. To address that, we add a database trigger to each collection…

Add Database Triggers

We need to create two Realm Functions—one to copy/transfer documents to ChatMessageV2, and one to copy documents to ChatMessage.

From the “Functions” section of the Realm UI, click “Create New Function”:

Name the function copyToChatMessageV2. Set the authentication method to “System”—this will circumvent any access permissions on the ChatMessageV2 collection. Ensure that the “Private” switch is turned on—that means that the function can be called from a trigger, but not directly from a frontend app. Click “Save”:

Paste this code into the function editor and save:

exports = function (changeEvent) {
   const db = context.services.get("mongodb-atlas").db("RChat");


   if (changeEvent.operationType === "delete") {
     return db.collection("ChatMessageV2").deleteOne({ _id: changeEvent.documentKey._id });
   }


   const author = changeEvent.fullDocument.author ? changeEvent.fullDocument.author : "Unknown";
   const pipeline = [
     { $match: { _id: changeEvent.documentKey._id } },
     {
       $addFields: {
         author: author,
       }
     },
     { $merge: "ChatMessageV2" }];


   return db.collection("ChatMessage").aggregate(pipeline);
};

This function will receive a ChatMessage document from our trigger. If the operation that triggered the function is a delete, then this function deletes the matching document from ChatMessageV2. Otherwise, the function either copies author from the incoming document or sets it to “Unknown” before writing the transformed document to ChatMessageV2. We could initialize author to any string, but I’ve used “Unknown” to tell the user that we don’t know who the author was.

Create the copyToChatMessage function in the same way:

exports = function (changeEvent) {
   const db = context.services.get("mongodb-atlas").db("RChat");


   if (changeEvent.operationType === "delete") {
     return db.collection("ChatMessage").deleteOne({ _id: changeEvent.documentKey._id })
   }
    const pipeline = [
     { $match: { _id: changeEvent.documentKey._id } },
     { $merge: "ChatMessage" }]
   return db.collection("ChatMessageV2").aggregate(pipeline);
};

The final change needed to the backend Realm application is to add database triggers that invoke these functions.

From the “Triggers” section of the Realm UI, click “Add a Trigger”:

Configure the ChatMessageProp trigger as shown:

Repeat for ChatMessageV2Change:

If you paused sync in the previous section, then you can now unpause it.

Updating the iOS RChat App

We want to ensure that users still running the old version of the app can continue to exchange messages with users running the latest version.

Existing versions of RChat will continue to work. They will create ChatMessage objects which will get synced to the ChatMessage Atlas collection. The database triggers will then copy/transform the document to the ChatMessageV2 collection.

We now need to create a new version of the app that works with documents from the ChatMessageV2 collection. We’ll cover that in this section.

Recall that we set title to ChatMessageV2 in the partner collection’s schema. That means that to sync with that collection, we need to rename the ChatMessage class to ChatMessageV2 in the iOS app.

Changing the name of the class throughout the app is made trivial by Xcode.

Open ChatMessage.swift and right-click on the class name (ChatMessage), select “Refactor” and then “Rename…”:

The final step is to make the author field mandatory. Remove the ? from the author attribute to make it non-optional:

class ChatMessageV2: Object, ObjectKeyIdentifiable {
   @Persisted(primaryKey: true) var _id = UUID().uuidString
   @Persisted var partition = "" // "conversation=<conversation-id>"
   @Persisted var author: String
   ...
}

Conclusion

Modifying a Realm schema is a little more complicated when you’re using Realm Sync for a deployed app. You’ll have end users who are using older versions of the schema, and those apps need to continue to work.

Fortunately, the most common schema changes (adding or removing fields) are additive. They simply require updates to the back end and iOS schema.

Things get a little trickier for destructive changes, such as changing the type or optionality of an existing field. For these cases, you need to create and maintain a partner collection to avoid loss of data or service for your users.

This article has stepped through how to handle both additive and destructive schema changes, allowing you to add new features or fix issues in your apps without impacting users running older versions of your app.

Remember, you can find all of the code for this post in the RChat repo under these branches:

• Starting point
• Upgrade #1
• Upgrade #2

If you’re looking to upgrade the Realm schema for an iOS app that isn’t using Realm Sync, then refer to the previous post in this series.

If you have any questions or comments on this post (or anything else Realm-related), then please raise them on our community forum. To keep up with the latest Realm news, follow @realm on Twitter and join the Realm global community.