Archive for January 26, 2022

Realm-Swift Type Projections

Introduction

Realm natively provides a broad set of data types, including Bool, Int, Float, Double, String, Date, ObjectID, List, Mutable Set, enum, Map, …

But, there are other data types that many of your iOS apps are likely to use. As an example, if you’re using Core Graphics, then it’s hard to get away without using types such as CGFloat, CGPoint, etc. When working with SwiftUI, you use the Color struct when working with colors.

A typical design pattern is to persist data using types natively supported by Realm, and then use a richer set of types in your app. When reading data, you add extra boilerplate code to convert them to your app’s types. When persisting data, you add more boilerplate code to convert your data back into types supported by Realm.

That works fine and gives you complete control over the type conversions. The downside is that you can end up with dozens of places in your code where you need to make the conversion.

Type projections still give you total control over how to map a CGPoint into something that can be persisted in Realm. But, you write the conversion code just once and then forget about it. The Realm-Swift SDK will then ensure that types are converted back and forth as required in the rest of your app.

The Realm-Swift SDK enables this by adding two new protocols that you can use to extend any Swift type. You opt whether to implement CustomPersistable or the version that’s allowed to fail (FailableCustomPersistable):

protocol CustomPersistable {
   associatedtype PersistedType
   init(persisted: PersistedType)
   var persistableValue: PersistedType { get }
}
protocol FailableCustomPersistable {
   associatedtype PersistedType
   init?(persisted: PersistedType)
   var persistableValue: PersistedType { get }
}

In this post, I’ll show how the Realm-Drawing app uses type projections to interface between Realm and Core Graphics.

Prerequisites

The Realm-Drawing App

Realm-Drawing is a simple, collaborative drawing app. If two people log into the app using the same username, they can work on a drawing together. All strokes making up the drawing are persisted to Realm and then synced to all other instances of the app where the same username is logged in.

Animated screen captures showing two iOS devices working on the same drawing

It’s currently iOS-only, but it would also sync with any Android drawing app that is connected to the same Realm back end.

Using Type Projections in the App

The Realm-Drawing iOS app uses three types that aren’t natively supported by Realm:

  • CGFloat
  • CGPoint
  • Color (SwiftUI)

In this section, you’ll see how simple it is to use type projections to convert them into types that can be persisted to Realm and synced.

Realm Schema (The Model)

An individual drawing is represented by a single Drawing object:

class Drawing: Object, ObjectKeyIdentifiable {
   @Persisted(primaryKey: true) var _id: ObjectId
   @Persisted var name = UUID().uuidString
   @Persisted var lines = RealmSwift.List<Line>()
}

A Drawing contains a List of Line objects:

class Line: EmbeddedObject, ObjectKeyIdentifiable {
   @Persisted var lineColor: Color
   @Persisted var lineWidth: CGFloat = 5.0
   @Persisted var linePoints = RealmSwift.List<CGPoint>()
}

It’s the Line class that uses the non-Realm-native types.

Let’s see how each type is handled.

CGFloat

I extend CGFloat to conform to Realm-Swift’s CustomPersistable protocol. All I needed to provide was:

  • An initializer to convert what’s persisted in Realm (a Double) into the CGFloat used by the model
  • A method to convert a CGFloat into a Double:
extension CGFloat: CustomPersistable {
   public typealias PersistedType = Double
   public init(persistedValue: Double) { self.init(persistedValue) }
   public var persistableValue: Double { Double(self) }
}

The view can then use lineWidth from the model object without worrying about how it’s converted by the Realm SDK:

context.stroke(path, with: .color(line.lineColor),
   style: StrokeStyle(
       lineWidth: line.lineWidth,
       lineCap: .round, l
       ineJoin: .round
   )
)

CGPoint

CGPoint is a little trickier, as it can’t just be cast into a Realm-native type. CGPoint contains the x and y coordinates for a point, and so, I create a Realm-friendly class (PersistablePoint) that stores just that—x and y values as Doubles:

public class PersistablePoint: EmbeddedObject, ObjectKeyIdentifiable {
   @Persisted var x = 0.0
   @Persisted var y = 0.0

   convenience init(_ point: CGPoint) {
       self.init()
       self.x = point.x
       self.y = point.y
   }
}

I implement the CustomPersistable protocol for CGPoint by mapping between a CGPoint and the x and y coordinates within a PersistablePoint:

extension CGPoint: CustomPersistable {
   public typealias PersistedType = PersistablePoint   
   public init(persistedValue: PersistablePoint) { self.init(x: persistedValue.x, y: persistedValue.y) }
   public var persistableValue: PersistablePoint { PersistablePoint(self) }
}

SwiftUI.Color

Color is made up of the three RGB components plus the opacity. I use the PersistableColor class to persist a representation of Color:

public class PersistableColor: EmbeddedObject {
   @Persisted var red: Double = 0
   @Persisted var green: Double = 0
   @Persisted var blue: Double = 0
   @Persisted var opacity: Double = 0

   convenience init(color: Color) {
       self.init()
       if let components = color.cgColor?.components {
           if components.count >= 3 {
               red = components[0]
               green = components[1]
               blue = components[2]
           }
           if components.count >= 4 {
               opacity = components[3]
           }
       }
   }
}

The extension to implement CustomPersistable for Color provides methods to initialize Color from a PersistableColor, and to generate a PersistableColor from itself:

extension Color: CustomPersistable {
   public typealias PersistedType = PersistableColor

   public init(persistedValue: PersistableColor) { self.init(
       .sRGB,
       red: persistedValue.red,
       green: persistedValue.green,
       blue: persistedValue.blue,
       opacity: persistedValue.opacity) }

   public var persistableValue: PersistableColor {
       PersistableColor(color: self)
   }
}

The view can then use selectedColor from the model object without worrying about how it’s converted by the Realm SDK:

context.stroke(
   path,
   with: .color(line.lineColor),
   style: StrokeStyle(lineWidth:
   line.lineWidth,
   lineCap: .round,
   lineJoin: .round)
)

Conclusion

Type projections provide a simple, elegant way to convert any type to types that can be persisted and synced by Realm.

It’s your responsibility to define how the mapping is implemented. After that, the Realm SDK takes care of everything else.

Please provide feedback and ask any questions in the Realm Community Forum.





Introducing Flexible Sync (Preview) – The Next Iteration of Realm Sync

We are excited to announce the public preview of our next version of Realm Sync: Flexible Sync. This new method of syncing puts the power into the hands of the developer. Now, developers can get more granular control over the data synced to user applications with intuitive language-native queries and hierarchical permissions.

Introduction

Prior to launching the general availability of Realm Sync in February 2021, the Realm team spent countless hours with developers learning how they build best-in-class mobile applications. A common theme emerged—building real-time, offline-first mobile apps require an overwhelming amount of complex, non-differentiating work.

Our first version of Realm Sync addressed this pain by abstracting away offline-first, real-time syncing functionality using declarative APIs. It expedited the time-to-market for many developers and worked well for apps where data is static and compartmentalized, or where permissions rarely need to change. But for dynamic apps and complex use cases, developers still had to spend time creating workarounds instead of developing new features. With that in mind, we built the next iteration of Realm Sync: Flexible Sync. Flexible Sync is designed to help developers:

  • Get to market faster: Use intuitive, language-native queries to define the data synced to user applications instead of proprietary concepts.
  • Optimize real-time collaboration between users: Utilize object-level conflict-resolution logic.
  • Simplify permissions: Apply role-based logic to applications with an expressive permissions system that groups users into roles on a pe-class or collection basis.

Language-Native Querying

Flexible Sync’s query-based sync logic is distinctly different from how Realm Sync operates today. The new structure is designed to more closely mirror how developers are used to building sync today—typically using GET requests with query parameters.

One of the primary benefits of Flexible Sync is that it eliminates all the time developers spend determining what query parameters to pass to an endpoint. Instead, the Realm APIs directly integrate with the native querying system on the developer’s choice of platform—for example, a predicate-based query language for iOS, a Fluent query for Android, a string-based query for Javascript, and a LINQ query for .NET.

Under the hood, the Realm Sync thread sends the query to MongoDB Realm (Realm’s cloud offering). MongoDB Realm translates the query to MongoDB’s query language and executes the query against MongoDB Atlas. Atlas then returns the resulting documents. Those documents are then translated into Realm objects, sent down to the Realm client, and stored on disk. The Realm Sync thread keeps a queue of any changes made locally to synced objects—even when offline. As soon as connectivity is reestablished, any changes made to the server-side or client-side are synced down using built-in granular conflict resolution logic. All of this occurs behind the scenes while the developer is interacting with the data. This is the part we’ve heard our users describe as “magic.”

Flexible Sync also enables much more dynamic queries, based on user inputs. Picture a home listing app that allows users to search available properties in a certain area. As users define inputs—only show houses in Dallas, TX that cost less than $300k and have at least three bedrooms—the query parameters can be combined with logical ANDs and ORs to produce increasingly complex queries, and narrow down the search result even further. All query results are combined into a single realm file on the client’s device, which significantly simplifies code required on the client-side and ensures changes to data are synced efficiently and in real time.

Swift

// Set your Schema
class Listing: Object {
    @Persisted(primaryKey: true) var _id: ObjectId
    @Persisted var location: String
    @Persisted var price: Int
    @Persisted var bedrooms: Int
}

// Configure your App and login
let app = App(id: "XXXX")
let user = try! await app.login(credentials:
            .emailPassword(email: "email", password: "password"))

// Set the new Flexible Sync Config and open the Realm
let config = user.flexibleSyncConfiguration()
let realm = try! await Realm(configuration: config, downloadBeforeOpen: .always)

// Create a Query and Add it to your Subscriptions
let subscriptions = realm.subscriptions

try! await subscriptions.write {
    subscriptions.append(QuerySubscription<Listing>(name: "home-search") {
        $0.location == "dallas" && $0.price < 300000 && $0.bedrooms >= 3
    })
}

// Now query the local realm and get your home listings - output is 100 listings
// in the results
print(realm.objects(Listing.self).count)

// Remove the subscription - the data is removed from the local device but stays
// on the server
try! await subscriptions.write {
    subscriptions.remove(named: "home-search")
}

// Output is 0 - listings have been removed locally
print(realm.objects(Listing.self).count)

Kotlin

// Set your Schema
open class Listing: ObjectRealm() {
  @PrimaryKey
  @RealmField("_id")
  var id: ObjectId
  var location: String = ""
  var price: Int = 0
  var bedrooms: Int = 0
}

// Configure your App and login
val app = App("<YOUR_APP_ID_HERE>")
val user = app.login(Credentials.emailPassword("email", "password"))

// Set the new Flexible Sync Config and open the Realm
let config = SyncConfiguration.defaultConfig(user)
let realm = Realm.getInstance(config)

// Create a Query and Add it to your Subscriptions
val subscriptions = realm.subscriptions
subscriptions.update { mutableSubscriptions ->
   val sub = Subscription.create(
      "home-search", 
      realm.where<Listing>()
         .equalTo("location", "dallas")
         .lessThan("price", 300_000)
         .greaterThanOrEqual("bedrooms", 3)
   )
   mutableSubscriptions.add(subscription)
}

// Wait for server to accept the new subscription and download data
subscriptions.waitForSynchronization()
realm.refresh()

// Now query the local realm and get your home listings - output is 100 listings 
// in the results
val homes = realm.where<Listing>().count()

// Remove the subscription - the data is removed from the local device but stays 
// on the server
subscriptions.update { mutableSubscriptions ->
   mutableSubscriptions.remove("home-search")
}
subscriptions.waitForSynchronization()
realm.refresh()

// Output is 0 - listings have been removed locally
val homes = realm.where<Listing>().count()

.NET

// Set your Schema
class Listing: RealmObject
{
    [PrimaryKey, MapTo("_id")]
    public ObjectId Id { get; set; }
    public string Location { get; set; }
    public int Price { get; set; }
    public int Bedrooms { get; set; }
}

// Configure your App and login
var app = App.Create(YOUR_APP_ID_HERE);
var user = await app.LogInAsync(Credentials.EmailPassword("email", "password"));

// Set the new Flexible Sync Config and open the Realm
var config = new FlexibleSyncConfiguration(user);
var realm = await Realm.GetInstanceAsync(config);

// Create a Query and Add it to your Subscriptions
var dallasQuery = realm.All<Listing>().Where(l => l.Location == "dallas" && l.Price < 300_000 && l.Bedrooms >= 3);
realm.Subscriptions.Update(() =>
{
    realm.Subscriptions.Add(dallasQuery);
});

await realm.Subscriptions.WaitForSynchronizationAsync();

// Now query the local realm and get your home listings - output is 100 listings
// in the results
var numberOfListings = realm.All<Listing>().Count();

// Remove the subscription - the data is removed from the local device but stays
// on the server

realm.Subscriptions.Update(() =>
{
    realm.Subscriptions.Remove(dallasQuery);
});

await realm.Subscriptions.WaitForSynchronizationAsync();

// Output is 0 - listings have been removed locally
numberOfListings = realm.All<Listing>().Count();

JavaScript

import Realm from "realm";

// Set your Schema
const ListingSchema = {
  name: "Listing",
  primaryKey: "_id",
  properties: {
    _id: "objectId",
    location: "string",
    price: "int",
    bedrooms: "int",
  },
};

// Configure your App and login
const app = new Realm.App({ id: YOUR_APP_ID_HERE });
const credentials = Realm.Credentials.emailPassword("email", "password");
const user = await app.logIn(credentials);

// Set the new Flexible Sync Config and open the Realm
const realm = await Realm.open({
  schema: [ListingSchema],
  sync: { user, flexible: true },
});

// Create a Query and Add it to your Subscriptions
await realm.subscriptions.update((mutableSubscriptions) => {
  mutableSubscriptions.add(
    realm
      .objects(ListingSchema.name)
      .filtered("location = 'dallas' && price < 300000 && bedrooms = 3", {
        name: "home-search",
      })
  );
});

// Now query the local realm and get your home listings - output is 100 listings
// in the results
let homes = realm.objects(ListingSchema.name).length;

// Remove the subscription - the data is removed from the local device but stays
// on the server
await realm.subscriptions.update((mutableSubscriptions) => {
  mutableSubscriptions.removeByName("home-search");
});

// Output is 0 - listings have been removed locally
homes = realm.objects(ListingSchema.name).length;

Optimizing for Real-Time Collaboration

Flexible Sync also enhances query performance and optimizes for real-time user collaboration by treating a single object or document as the smallest entity for synchronization. Flexible Sync allows for Sync Realms to more efficiently share data and for conflict resolution to incorporate changes faster and with less data transfer.

For example, you and a fellow employee are analyzing the remaining tasks for a week. Your coworker wants to see all of the time-intensive tasks remaining (workunits &gt; 5), and you want to see all the tasks you have left for the week (owner == ianward). Your queries will overlap where workunits &gt; 5 and owner == ianward. If your coworker notices one of your tasks is marked incorrectly as 7 workunits and changes the value to 6, you will see the change reflected on your device in real time. Under the hood, the merge algorithm will only sync the changed document instead of the entire set of query results increasing query performance.

Venn diagram showing that 2 different queries can share some of the same documents

Permissions

Whether it’s a company’s internal application or an app on the App Store, permissions are required in almost every application. That’s why we are excited by how seamless Flexible Sync makes applying a document-level permission model when syncing data—meaning synced documents can be limited based on a user’s role.

Consider how a sales organization uses a CRM application. An individual sales representative should only be able to access her own sales pipeline while her manager needs to be able to see the entire region’s sales pipeline. In Flexible Sync, a user’s role will be combined with the client-side query to determine the appropriate result set. For example, when the sales representative above wants to view her deals, she would send a query where opportunities.owner == "EmmaLullo" but when her manager wants to see all the opportunities for their entire team, they would query with opportunities.team == “West”. If a user sends a much more expansive query, such as querying for all opportunities, then the permissions system would only allow data to be synced for which the user had explicit access.

{
  "Opportunities": {
    "roles": [
        {
                name: "manager", 
                applyWhen: { "%%user.custom_data.isSalesManager": true},
                read: {"team": "%%user.custom_data.teamManager"}
                write: {"team": "%%user.custom_data.teamManager"}
            },
        {
                name: "salesperson",
                applyWhen: {},
                read: {"owner": "%%user.id"}
                write: {"owner": "%%user.id"}
        }
    ]
  },
{
  "Bookings": {
    "roles": [
        {
                name: "accounting", 
                applyWhen: { "%%user.custom_data.isAccounting": true},
                read: true,
                write: true
            },
        {
                name: "sales",
                applyWhen: {},
                read: {"%%user.custom_data.isSales": true},
                write: false
        }
    ]
  }

Looking Ahead

Ultimately, our goal with Flexible Sync is to deliver a sync service that can fit any use case or schema design pattern imaginable without custom code or workarounds. And while we are excited that Flexible Sync is now in preview, we’re nowhere near done.

The Realm Sync team is planning to bring you more query operators and permissions integrations over the course of 2022. Up next we are looking to expose array operators and enable querying on embedded documents, but really, we look to you, our users, to help us drive the roadmap. Submit your ideas and feature requests to our feedback portal and ask questions in our Community forum. Happy building!