LiveQuery does not work with PFObjects secured with an ACL & when the where key has a Pointer

[tsanthosh]

First, thanks to the Parse team for making Parse open source and enhancing it constantly.

Environment:

• parse: 1.8.5
• parse-server: 2.2.11
• node: 4.4.5

Parse-server index.js:

...
var api = new ParseServer({
  databaseURI: databaseUri || 'mongodb://localhost:27017/xxxx-dev',
  cloud: process.env.CLOUD_CODE_MAIN || __dirname + '/cloud/main.js',
  appId: process.env.APP_ID || 'xxxx',
  masterKey: process.env.MASTER_KEY || 'xxx',
  serverURL: process.env.SERVER_URL || 'http://localhost:1337/parse',
  filesAdapter: new S3Adapter(xxxx),
  liveQuery: {
    classNames: ["Account", "GUEvent"]
  }
});
...
ParseServer.createLiveQueryServer(httpServer);

iOS Client:

class viewController: UIViewController {
    private var subscription: Subscription<GUEvent>!

    override func viewDidLoad() {
        super.viewDidLoad()

        subscription = GUEvent.query()!.whereKey("account", equalTo: accountObject).subscribe()
        subscription.handleEvent { query, event in
             print("Handling Event")
        }
    }
}

Live Query does not work:

• When the objects in GUEvent are secured with an ACL but works when the objects are publicly readable & writable.
• When the account is a pointer, I get *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: 'Invalid type in JSON write'.

So far Live Query works:

• When the objects in GUEvent are publicly readable and writable.
• When the where statement does not contain a pointer.

Any help would be greatly appreciated.

[tsanthosh]

I solved the ACL issue by passing the sessionToken to the subscribe method of the live query protocol.

>> In Operation.swift
internal enum ClientOperation {
    case Connect(applicationId: String, sessionToken: String)
    case Subscribe(requestId: Client.RequestId, query: PFQuery, **sessionToken: String**)
    case Unsubscribe(requestId: Client.RequestId)

    var JSONObjectRepresentation: [String : AnyObject] {
        switch self {
        case .Connect(let applicationId, let sessionToken):
            return [ "op": "connect", "applicationId": applicationId, "sessionToken": sessionToken ]

        case .Subscribe(let requestId, let query, **let sessionToken**):
            return [ "op": "subscribe", "requestId": requestId.value, "query": Dictionary<String, AnyObject>(query: query), **"sessionToken": sessionToken** ]

        case .Unsubscribe(let requestId):
            return [ "op": "unsubscribe", "requestId": requestId.value ]
        }
    }
}

>> In Client.swift & ClientPrivate.swift
self.sendOperationAsync(.Subscribe(requestId: $0.requestId, query: $0.query, sessionToken: PFUser.currentUser()!.sessionToken!))

This worked and I was able to get changes about objects which have been secured with an ACL.

[tsanthosh]

I also solved the pointer issue by replacing:

if let conditions: [String:AnyObject] = queryState?.valueForKey("conditions") as? [String:AnyObject] {
    print(conditions)
    self["where"] = conditions as? Value
}

with

if let rawConditions: [String:AnyObject] = queryState?.valueForKey("conditions") as? [String:AnyObject] {
    var formattedConditions = [String: AnyObject]()
    let keys = rawConditions.keys
    for eachKey in keys {
        if let pointer = rawConditions[eachKey] as? PFObject {
            formattedConditions[eachKey] = ["__type":"Pointer","className": pointer.parseClassName,"objectId": pointer.objectId!]
        } else {
            formattedConditions[eachKey] = rawConditions[eachKey]
        }
    }
    self["where"] = formattedConditions as? Value
}

in QueryEncoder.swift

At this stage I was super excited that I for the first time could do a pull request to update this repo. But what I found was PFObject received by the subscription handler was not readable i.e. instead of getting an object in the following format.

<Attendee: 0x7f8afce40440, objectId: ptnVOC4tKE, localId: (null)> {
    ACL = "<PFACL: 0x7f8afcebd670>";
    attendeeId = 585857109;
    barcode = 463895634585857109001;
    checkedIn = 0;
    customFields =     (
        "<CustomField: 0x7f8afa529840, objectId: 8Eri3BCtnf, localId: (null)>",
        "<CustomField: 0x7f8afa52aea0, objectId: rX21uQozak, localId: (null)>"
    );
    event = "<GUEvent: 0x7f8afa6a6720, objectId: Xsjhm2GrSZ, localId: (null)>";
    firstName = xxxxx;
    lastName = xxx;
}

I get.

<Attendee: 0x7f8afa6add50, objectId: sBA6bA6iwo, localId: (null)> {
    ACL =     {
        w6Pc41N1jN =         {
            read = 1;
            write = 1;
        };
    };
    "__type" = Object;
    attendeeId = 585857110;
    barcode = 463895634585857110001;
    checkedIn = 1;
    className = Attendee;
    createdAt = "2016-06-08T04:09:09.243Z";
    customFields =     (
                {
            "__type" = Pointer;
            className = CustomField;
            objectId = u62YrQ1QQL;
        },
                {
            "__type" = Pointer;
            className = CustomField;
            objectId = IXHYy0EQQn;
        }
    );
    event =     {
        "__type" = Pointer;
        className = GUEvent;
        objectId = Xsjhm2GrSZ;
    };
    firstName = xxxx;
    lastName = xxxx;
    updatedAt = "2016-06-08T09:06:06.709Z";
}

This looks like the results are from a REST API.

I then thought the above problem was due to me changing the QueryEncoder.swift but that was not the case. Even without using a pointer, I found that the PFObjects which were being downloaded after an event were not suitable for reading in iOS.

So, I am still stuck with this problem. Was super excited about Live Queries but now I am confused.

Any help will be much appreciated.

[vespakoen]

It looks like the results you get there in the last code block have pointers to other objects.
Did you expect the to be the full objects? in that case, did you use "includeKey" on the query?

I would love to see your pull request anyways, that way other people can test it out / give feedback =)

[agordeev]

Hi @tsanthosh ! Thanks for this research, I'm facing the same issue (parse-community/parse-server#2227). Did you make a pull request yet?

[agordeev]

Hi @tsanthosh ! I tried your method of passing sessionToken, but it doesn't seem working. The app doesn't get any event. Can you please check if your method is working for you? May be I missed something.

[ideastouch]

Hi Guys,
I think that the solution of @tsanthosh is a great start but fail when the query has some constraint as for example Less Than, etc (see link).
Therefore I added a block that it calls itself if necessary.

My change of

        if let conditions: [String:AnyObject] = queryState?.valueForKey("conditions") as? [String:AnyObject] {
            print(conditions)
            self["where"] = conditions as? Value
        }

is

        let objectDictionary = { (object:PFObject)->[String:String] in
            return ["__type":"Pointer","className": object.parseClassName,"objectId": object.objectId!] }
        var valueBlock:((AnyObject)->AnyObject?)!
        valueBlock = {(value:AnyObject)->AnyObject? in
            if let value = value as? PFObject {
                return objectDictionary(value) }
            if let value = value as? [String:AnyObject] {
                guard let (key,value) = value.first else {
                    fatalError("Value is bad formatted.")
                    return nil}
                guard let optional = valueBlock(value) else {
                    fatalError("Value is bad formatted.")
                    return nil}
                return [key: optional] }
            return value }

        if let rawConditions: [String:AnyObject] = queryState?.valueForKey("conditions") as? [String:AnyObject] {
            var formattedConditions = [String: AnyObject]()
            let keys = rawConditions.keys
            for (key,value) in rawConditions {
                formattedConditions[key] = valueBlock(value)
            } 
            self["where"] = formattedConditions as? Value
        }

I'm not sure if my solution when the guard fails is good, probably needs to be improved.
Anyway, if the guard's doesn't fail the query seems good formatted.
I wish our problems where solved with my solution above but looks like there is more issues.

The function parseObject<T: PFObject>(objectDictionary: [String:AnyObject]) throws -> T at ClientPrivate.swift doesn't take in account that the objectDictionary may have values that can be Parse Object. Therefore again we may use recursive calls.
I replaced the code

    let parseObject = T(withoutDataWithClassName: parseClassName, objectId: objectId)
    objectDictionary.filter { key, _ in
        key != "parseClassName" && key != "objectId"
        }.forEach { key, value in
            else { parseObject[key] = value }
    }

by

    let object = T(withoutDataWithClassName: parseClassName, objectId: objectId)
    try objectDictionary.filter { key, _ in
        key != "parseClassName" && key != "objectId"
        }.forEach { key, value in
            if let valueDictionary = value as? [String:AnyObject] {
                try parseObject(valueDictionary) }
            else { object[key] = value }
    }

Finally, looks like Live Query Server ignores where options in queries that has a constraint. In my case I tried using the constraint whereKey(String,notEqualTo:AnyObject). And the result was that I received everything.
I didn't check yet where the problem is, but I will try find it tomorrow.

[ideastouch]

I can not check what I said in my previous comments about what is going on with my request subscription, why the Server doesn't take in count the "notEqualTo" constraint. The request seems configured well, see below:

{
   "requestId":1,
   "op":"subscribe",
   "query":{"className":"Chat",
   "where":{
      "sender":{"$ne":{"className":"_User","objectId":"PQauDXIuWZ","__type":"Pointer"}},
      "appointment":{"className":"Appointment","objectId":"Z7J9CfWCZA","__type":"Pointer"}}}}

I can not check it because I can not succeed config the Parse Live Query with logs in VERBOSE mode. My configurations are as below:

var liveQuery = {
   logLevel: 'VERBOSE',
   classNames: ["Chat", "PFUser", "Appointment"] }
var api = new ParseServer({
  databaseURI: process.env.MONGODB_URI,
  cloud: process.env.CLOUD_CODE_MAIN,
  appId: process.env.APP_ID,
  clientKey: process.env.CLIENT_ID,
  masterKey: process.env.MASTER_KEY,
  serverURL: process.env.SERVER_URL,
  fileKey:  process.env.FILE_KEY,
  restAPIKey: process.env.RESTAPI_KEY,
  javascriptKey: process.env.JS_KEY,
  publicServerURL: process.env.SERVER_URL,
  emailAdapter: emailAdapter,
  liveQuery: liveQuery });

var port = process.env.PORT || process.env.PORT_SRVR

app.set('port', process.env.PORT || process.env.PORT_SRVR);
var server = require('http').createServer(app);
server.listen(app.get('port'), function () {
    console.log("Express server listening on port " + app.get('port')); });

// This will enable the Live Query real-time server
ParseServer.createLiveQueryServer(server, liveQuery);

Any help will be very welcome.

Best!

[andrewjedi]

When will this be fixed :(

[Treverr]

Do we have anymore information on this? A lot of my databases use pointers and makes the Live Query quite unusable.

[mmowris]

@Treverr , likely be a while. Doesn't look like this repo is being actively supported :(

[protspace]

The same for me. Result object from LiveQuery looks like REST API response object:

<Activity: 0x6080004ad8c0, objectId: E73ZyCFYGc, localId: (null)> {
    "__type" = Object;
    author =     {
        "__type" = Pointer;
        className = "_User";
        objectId = mEWnXHChMO;
    };
    className = Activity;
    createdAt = "2016-10-21T11:51:57.133Z";
    date =     {
        "__type" = Date;
        iso = "2016-10-21T15:01:00.000Z";
    };
    description = "";
    duration = 0;
    gender = 2;
    level = 1;
    location =     {
        "__type" = GeoPoint;
        latitude = "50.42923516132909";
        longitude = "30.51982775333386";
    };
    maxUsers = 20;
    name = 0;
    pendingUsers =     (
                {
            "__type" = Pointer;
            className = "_User";
            objectId = RGBrnHLBeC;
        }
    );
    updatedAt = "2016-10-22T15:31:24.366Z";
}

[flovilmart]

Can you try with the PR #76 ?

You can use it directly with CocoaPods (Swift3)

pod 'ParseLiveQuery', git: 'https://github.com/parseplatform/ParseLiveQuery-iOS-OSX', branch: "fix-object-decoding"

[protspace]

Thanks, @flovilmart. It works. Now returns correct PFObject

[protspace]

But includeKey is still not working for me.

[flovilmart]

The includedKeys is likely a problem with parse server as we don't perform back inclusion when notifying for a change on a certain query. We match on the query constraints not on the additional portion like selectKeys, order étc...

[ideastouch]

I think that the issues at files BoltsHelpers.swift and ClientPrivate.swift where fixed on tag '1.1.0'. On the other side the file QueryEncoder.swift still has not any change related the issue at serialization time with queries that has for example Less Than, see link. In this case I still use my fix, see below:

extension Dictionary where Key: ExpressibleByStringLiteral, Value: AnyObject {
    init<T>(query: PFQuery<T>) where T: PFObject {
        self.init()
        let queryState = query.value(forKey: "state") as AnyObject?
        if let className = queryState?.value(forKey: "parseClassName") {
            self["className"] = className as? Value
        }
       //  Code Before
       //  if let conditions: [String:AnyObject] = queryState?.value(forKey: "conditions") as? [String:AnyObject] {
       //      self["where"] = conditions.encodedQueryDictionary as? Value
       //  }
        let objectDictionary = { (object:PFObject)->[String:String] in
            return ["__type":"Pointer","className": object.parseClassName,"objectId": object.objectId!] }

        // Block valueBlock is my proposed fix
        var valueBlock:((AnyObject)->AnyObject?)!
        valueBlock = {(value:AnyObject)->AnyObject? in
            if let value = value as? PFObject {
                return objectDictionary(value) as? AnyObject }
            if let value = value as? [String:AnyObject] {
                guard let (key,value) = value.first else {
                    fatalError("Value is bad formatted.")
                    return nil}
                guard let optional = valueBlock(value) else {
                    fatalError("Value is bad formatted.")
                    return nil}
                return [key: optional]  as? AnyObject }
            return value }

        if let rawConditions: [String:AnyObject] = queryState?.value(forKey:"conditions") as? [String:AnyObject] {
            var formattedConditions = [String: AnyObject]()
            let keys = rawConditions.keys
            for (key,value) in rawConditions {
                formattedConditions[key] = valueBlock(value)
            }
            self["where"] = formattedConditions as? Value
        }
    }
}

[flovilmart]

Did you open a Pull Request with the fix for the query encoding?

[danielchangsoojones]

I redirected my pod to the fix-object-decoding branch, and when I run my query query.whereKey("sender", notEqualTo: User.current()!), I am still getting the error "Invalid type in JSON write". I am kind of lost at this point, are there more necessary steps to take than just redirecting the Cocoapod? Thanks in advance.

[flovilmart]

the fix-object-decoding don't fix the query encoding, but the object decoding upon reception.

[danielchangsoojones]

Oh, so then is it still not currently possible to do a query on a pointer column (e.g. query.whereKey("sender", notEqualTo: User.current()!) ) ?

[flovilmart]

it doesn't seem like it. PR are welcome

[ShawnBaek]

@protspace Did you solve this problem? I got an exactly same problems.

[magneticrob]

@yoshiboarder We hacked our way around it by creating a new column with the objectId of our pointer in text ¯\_(ツ)_/¯

[ShawnBaek]

@magneticrob good tip Thanks :)
Is this problem still not fixed?

[flovilmart]

The problem seems to be server side in that case. Can you open the issue there with clear steps to reproduce?

[ShawnBaek]

@flovilmart Open at #86

[ShawnBaek]

I think livequery is not support pointer object. Is it right?
I had tried update my pointer object but there is no events happened.

Here is my solution.

I use fix-object-decoding.

When I received live query event then called fetchIfNeeded() in event handler.

 do {
                try obj.postBy?.fetchIfNeeded()
                try obj.commentPointer?.fetchIfNeeded()
                
            } catch _ {
                print("There was an error")
            }

And I also use Cloud Function to receive events when changed Pointer Object. In my case, When I update PFUser then reset postBy in Post Class.

I hope this will help.

[protspace]

@yoshiboarder the only problem I still have is inconsistency between updated object arriving in LQ callback and current old object I have at this moment. Though objectId's of this objects are the same I have basically 2 different objects. SO what I do is just one extra step - query object with given objectId or fetching

[ShawnBaek]

@protspace Do you means like this?

<Post: 0x6080000b0800, objectId: w5qcfMCByF, localId: (null)> {
likeCount = 0;
postBy = "<PFUser: 0x6080000e3100, objectId: rgG7XfAYWj, localId: (null)>";
postImg = "<PFFile: 0x60800024c150>";
postText = nice2MeetYous1211;
sell = 0;
}

When I change postText then I got update event via live query but different.

<Post: 0x6100000b1be0, objectId: w5qcfMCByF, localId: (null)> {
likeCount = 0;
postBy = "<PFUser: 0x6100000e6900, objectId: rgG7XfAYWj, localId: (null)>";
postImg = "<PFFile: 0x61000025d1f0>";
postText = nice2MeetYous;
sell = 0;
}

As you can see..PFUser:0x6080000e3100 and PFFile: 0x60800024c150 has changed to
PFUser: 0x6100000e6900, PFFile: 0x61000025d1f0

---

In my case, I had called fetching.. not query object with id, before I get data. :)

[protspace]

my model is way mote complicated, but you got the main thing - updating "your" object from server manually

[ljs19923]

ACL with current user works, but ACL with Role seems to doesn't works.... it's normal ?

[ranhsd]

Hi , Any news regarding this issue?
The only way i manage to "solve" it is by execute additional call to the server with the objectId in order to get the full object but its not so efficient ..

[flovilmart]

This is an issue with parse-server no?

[ranhsd]

I guess so... according to your comments this is what i understand but anyway at the end the iOS client will use it... do you think we should move it to parse-server issues ?

[flovilmart]

Yes most probably, the client is quite 'dumb' and has no matter in this behavior

[chlebta]

@flovilmart 1.1.0 I just figured that pod install doesn't install the latest version I don't know why.
Thank you I used pod 'ParseLiveQuery', '~> 2.0' to install 2.0.0 version

[Charlesleonius]

@tsanthosh Could you include do a pull request with your fix to allow the live querying of ACL secured objects? This would be an incredibly useful feature.

[tsanthosh]

@Charlesleonius I believe live querying of ACL secured objects is fixed now thanks to @andrew8712. Try v2.0.0.

[frangulyan]

Role ACLs are not yet fixed, not working in my case...

[flovilmart]

What do you mean by that? Open an issue on Parse-server and properly fill the template please.

[frangulyan]

@flovilmart I actually opened one here - 4311

and then I found similar ones, like 106, 2018 and 349.

In #2018 Parse guys actually suggest to fix it ourselves and submit a pull request, but the most interesting is #349 where I have commented myself several months ago and totally forgot about it. There I found the reason and will update 4311 that I created earlier today.

Basic idea is the following - if _Role table is locked for master-key-only ACL then live query on objects protected by Role ACLs will fail. As a result I opened the Role table for public read and live queries started to work again, however I don't like this solution, want to keep Role table closed.

[flovilmart]

Please, if it’s an issue on Parse server open it with all the informations required to fix it, as I see you didn’t fill the issue template either.

[ghost]

We're in May 2018 and this is still an issue. Does anyone have a working fix for this?

[magneticrob]

@Xenero not a working fix, but a working workaround - create a new column with the objectID in text, and use that for your LiveQuery. See my comment further up in the thread. It's ugly, but it works.

[ghost]

@magneticrob thankyou for the suggestion, whilst this won't work for my instance because the data is extremely dynamic (i'm building a 'stories' system) and changes regularly I need something more robust.

This lead me to creating a function that returns me the query as an array of objectId's like below. For now it works in eliminating the error.

NSMutableArray *)collectiveFriendsArrayForCurrentUserAsObjects {
    //Create Friends Array
    NSMutableArray *collectiveFriendsArray = [[NSMutableArray alloc] init];
    
    //Check For Current User
    if ([PFUser currentUser]) {
        for (PFObject *friendObject in [[PFUser currentUser] objectForKey:@"friendsArray"]) {
            [collectiveFriendsArray addObject:[friendObject objectId]];
        }
        [collectiveFriendsArray addObject:(PFObject *)[[PFUser currentUser] objectId]];
    }
    
    return collectiveFriendsArray;
}

The issue now is that it is not returning the notifier of any events. Any idea why this is and how I can fix it?

[flovilmart]

@Xenero feel free to open a pull request with a fix on parse-server. Also the discussion as moved away from the original topic which was solved a while ago. Locking the conversation.