withinKilometers withinMiles withinRadians not working with Live Query #7480

PolSpock · 2021-08-04T12:07:52Z

New Issue Checklist

I am not disclosing a vulnerability.
I am not just asking a question.
I have searched through existing issues.
I can reproduce the issue with the latest version of Parse Server.

Issue Description

I want to retrieve object only in a specific radius of the user current Position.
I use Live Query to get when object enter, update, leave theses conditions in real time.

However when i write a query with withinKilometers / withinMiles / withinRadians the Live Query never triggered when the Object has his GeoPoint enter / update / leave the specific radius.

Steps to reproduce

Setup a parse-server with Live Query
Create a class with a GeoPoint column (in my example, my column is "location")
Create a client, i tried on Parse Flutter SDK and Parse Javascript SDK
Connect the client, make a Parse Query with a subscription based on withinKilometers / withinMiles / withinRadians conditions
My javascript example :

<html>
<head>
    <meta charset="utf-8">
    <title>Test</title>
    <script src="https://unpkg.com/parse/dist/parse.js"></script>
    <script>
        main();
        async function main() {
            Parse.initialize("xxxxxxxxxxxxx");
            Parse.serverURL = 'http://192.168.X.X:1337/parse'

            var query = new Parse.Query('TestObject');
            query.withinKilometers("location", new Parse.GeoPoint({latitude: 45, longitude: 2}), 1, false);
            var subscription = await query.subscribe();

            subscription.on('open', () => {
                console.log('subscription opened');
            });

            subscription.on('update', (object) => {
                console.log(object);
                console.log('object updated');
            });

            subscription.on('enter', (object) => {
                console.log(object);
                console.log('enter updated');
            });

            subscription.on('leave', (object) => {
                console.log(object);

                console.log('object left');
            });

            subscription.on('delete', (object) => {
                console.log(object);

                console.log('object deleted');
            });

            subscription.on('close', () => {
                console.log('subscription closed');
            });
        }
    </script>
</head>
<body>
    <p>Test</p>
</body>
</html>

Actual Outcome

My Live Query subscription is never triggered when the GeoPoint respect the subscription conditions.

If i replace the .subscription() by .find() the query worked fine, but this is not i want cause i need Live Query

Expected Outcome

I expect to get my TestObject when his location is update and still respect my subscription condition.

In my example : a TestObject with "location" Parse.GeoPoint({latitude: 45, longitude: 2}) must be return to client because the client have the same position and the withinKilometers is respected.
(see my example line query.withinKilometers("location", new Parse.GeoPoint({latitude: 45, longitude: 2}), 1, false);)

Failing Test Case / Pull Request

🤩 I submitted a PR with a fix and a test case.
🧐 I submitted a PR with a failing test case.

Environment

Server

Parse Server version: 4.5.0
Operating system: Windows 10.0.19043
Local or remote host (AWS, Azure, Google Cloud, Heroku, Digital Ocean, etc): local

Database

System (MongoDB or Postgres): MongoDB
Database version: 4.4.4
Local or remote host (MongoDB Atlas, mLab, AWS, Azure, Google Cloud, etc): Local

Clients

SDK JavaScript version: 3.3.0
SDK Flutter version: 3.1.0

Logs

No log

Regards !

The text was updated successfully, but these errors were encountered:

mtrezza · 2021-08-04T12:48:10Z

Thanks for reporting! Could you add a failing test to demonstrate the issue?

PolSpock · 2021-08-04T19:07:28Z

Hi, after new testing i found new elements :

With Parse JavasScript SDK :
If you give the last parameter of withinKilometers (sorted) : query.withinKilometers("location", new Parse.GeoPoint({ latitude: 45.0, longitude: 2.0 }), 2, false);
The request to register the subscription to the server will be :

{"op":"subscribe","requestId":1,"query":{"className":"TestObject","where":{"location":{"$geoWithin":{"$centerSphere":[[2,45],0.0003139224611520954]}}}}}

This not working

However, if you not specify the last parameter of withinKilometers : query.withinKilometers("location", new Parse.GeoPoint({ latitude: 45.0, longitude: 2.0 }), 2);
The subscription request to the server will be :

{"op":"subscribe","requestId":1,"query":{"className":"TestObject","where":{"location":{"$nearSphere":{"__type":"GeoPoint","latitude":45,"longitude":2},"$maxDistance":0.0003139224611520954}}}}

And this working

However with Parse Flutter SDK :
The following code :

ParseGeoPoint contraint = new ParseGeoPoint();
contraint.latitude = 45.0;
contraint.longitude = 2.0;

ParseObject truckObject = ParseObject("TestObject");

QueryBuilder<ParseObject> query = QueryBuilder<ParseObject>(truckObject)
  ..whereWithinKilometers("location", contraint, 2);

_subscription = await liveQuery.client.subscribe(query);

Generate the cURL request which not working too :

SubscribeMessage: {op: subscribe, requestId: 9, query: {className: TestObject, where: {location: {$nearSphere: {__type: GeoPoint, latitude: 45.0, longitude: 2.0}, $maxDistanceInKilometers: 2.0}}}, sessionToken: r:eeeeeeeeeeeeeeeeeeeeeeeeeeee}

So i see they are different result of request between Javascript SDK and Flutter SDK
$maxDistance parameter working but $maxDistanceInKilometers and $geoWithin":"$centerSphere doesn't

Test with Jasmine in /spec folder on parse-server repo :
Not working test :

  it('can subscribe to query and return object with withinKilometers with last parameter on update', async done => {
    await reconfigureServer({
      liveQuery: {
        classNames: ['TestObject'],
      },
      startLiveQueryServer: true,
      verbose: true,
      silent: false,
    });
    const object = new TestObject();
    const firstPoint = new Parse.GeoPoint({ latitude: 40.0, longitude: -30.0 });
    object.set({ location: firstPoint });
    await object.save();

    const query = new Parse.Query(TestObject);
    query.withinKilometers("location", new Parse.GeoPoint({ latitude: 40.0, longitude: -30.0 }), 2, false);
    const subscription = await query.subscribe();
    subscription.on('update', obj => {
      console.log("update");
      expect(obj.id).toBe(object.id);
      done();
    });

    const secondPoint = new Parse.GeoPoint({ latitude: 40.0, longitude: -30.0 });
    object.set({ location: secondPoint });
    await object.save();
  });

Working test :

 it('can subscribe to query and return object with withinKilometers without last parameter on update', async done => {
  await reconfigureServer({
    liveQuery: {
      classNames: ['TestObject'],
    },
    startLiveQueryServer: true,
    verbose: true,
    silent: false,
  });
  const object = new TestObject();
  const firstPoint = new Parse.GeoPoint({ latitude: 40.0, longitude: -30.0 });
  object.set({ location: firstPoint });
  await object.save();

  const query = new Parse.Query(TestObject);
  query.withinKilometers("location", new Parse.GeoPoint({ latitude: 40.0, longitude: -30.0 }), 2);
  const subscription = await query.subscribe();
  subscription.on('update', obj => {
    console.log("update");
    expect(obj.id).toBe(object.id);
    done();
  });

  const secondPoint = new Parse.GeoPoint({ latitude: 40.0, longitude: -30.0 });
  object.set({ location: secondPoint });
  await object.save();
});

EDITED :

On Parse Flutter SDK, if i override this method

  /// Returns an object with key point values near the point given and within the maximum distance given.
  void whereWithinKilometers(
      String column, ParseGeoPoint point, double maxDistance) {
    final double latitude = point.latitude;
    final double longitude = point.longitude;

    queries.add(MapEntry<String, dynamic>(_SINGLE_QUERY,
        '\"$column\":{\"\$nearSphere\":{\"__type\":\"GeoPoint\",\"latitude\":$latitude,\"longitude\":$longitude},\"\$maxDistanceInKilometers\":$maxDistance}'));
  }

By :

  /// Returns an object with key point values near the point given and within the maximum distance given.
  void whereWithinKilometers(
      String column, ParseGeoPoint point, double maxDistance) {
    final double latitude = point.latitude;
    final double longitude = point.longitude;

    queries.add(MapEntry<String, dynamic>(_SINGLE_QUERY,
        '\"$column\":{\"\$nearSphere\":{\"__type\":\"GeoPoint\",\"latitude\":$latitude,\"longitude\":$longitude},\"\$maxDistance\":$maxDistance}'));
  }

The override works : but you have to convert kilometers in maxDistance (see there)

So i really think they are a problem with Live Query and Parse Javascript SDK / Parse Flutter SDK parameters

EDITED 2 :
So i've seen in the parse-server source code these lines :

      // The SDKs don't seem to use these but they are documented in the
      // REST API docs.
      case '$maxDistanceInRadians':
        answer['$maxDistance'] = constraint[key];
        break;
      case '$maxDistanceInMiles':
        answer['$maxDistance'] = constraint[key] / 3959;
        break;
      case '$maxDistanceInKilometers':
        answer['$maxDistance'] = constraint[key] / 6371;
        break;

So $maxDistanceInKilometers $maxDistanceInMiles $maxDistanceInRadians must have to be converted to $maxDistance but it doesn't seems to work :/

EDITED 3 :
If you add log on these lines, the query.find() request will fire correctly the console.log("i'm here"); but nothing appear with a Live Query

     // The SDKs don't seem to use these but they are documented in the
     // REST API docs.
     case '$maxDistanceInRadians':
       answer['$maxDistance'] = constraint[key];
       break;
     case '$maxDistanceInMiles':
       answer['$maxDistance'] = constraint[key] / 3959;
       break;
     case '$maxDistanceInKilometers':
       console.log("i'm here");
       answer['$maxDistance'] = constraint[key] / 6371;
       break;

github-actions · 2021-08-27T02:09:43Z

🤖 Parsy

Thanks for opening this issue!

🚀 You can help us to fix this issue faster by opening a pull request with a failing test. See our Contribution Guide for how to make a pull request, or read our New Contributor's Guide if this is your first time contributing. In any case, feel free to ask if you have any questions.

I'm in beta, so forgive me if I'm still making mistakes.

mtrezza · 2021-08-27T11:59:18Z

@PolSpock Never mind, that message was because this issue was written with an old template. I guess the bot should only comment on newly opened issues.

cbaker6 · 2021-08-29T14:35:53Z

With Parse JavasScript SDK :
If you give the last parameter of withinKilometers (sorted) : query.withinKilometers("location", new Parse.GeoPoint({ latitude: 45.0, longitude: 2.0 }), 2, false);
The request to register the subscription to the server will be :

{"op":"subscribe","requestId":1,"query":{"className":"TestObject","where":{"location":{"$geoWithin":{"$centerSphere":[[2,45],0.0003139224611520954]}}}}}

This not working

However, if you not specify the last parameter of withinKilometers : query.withinKilometers("location", new Parse.GeoPoint({ latitude: 45.0, longitude: 2.0 }), 2);
The subscription request to the server will be :

{"op":"subscribe","requestId":1,"query":{"className":"TestObject","where":{"location":{"$nearSphere":{"__type":"GeoPoint","latitude":45,"longitude":2},"$maxDistance":0.0003139224611520954}}}}

And this working

I'm sure this has something to do with LiveQuery currently not supporting all query operators. More info here, specifically:

The query.where field is mandatory. It represents the condition of the ParseQuery the client subscribes to. The format of the where field is the same with ParseQuery's REST API format. You can check the detail here. Right now we support $lt, $lte, $gt, $gte, $ne, $in, $nin, $exists, $all, $regex, $nearSphere, $within and normal equal condition. Any unsupported conditions will be ignored.

Though there has been additional support added in #7113

You can see that $maxDistance is supported here:

parse-server/src/LiveQuery/QueryTools.js

Lines 305 to 311 in 1e0d408

    
           case '$nearSphere': 
        
             if (!compareTo || !object[key]) { 
        
               return false; 
        
             } 
        
             var distance = compareTo.radiansTo(object[key]); 
        
             var max = constraints.$maxDistance || Infinity; 
        
             return distance <= max;

While $centerSphere isn't:

parse-server/src/LiveQuery/QueryTools.js

Lines 336 to 340 in 1e0d408

    
           case '$geoWithin': { 
        
             const points = compareTo.$polygon.map(geoPoint => [geoPoint.latitude, geoPoint.longitude]); 
        
             const polygon = new Parse.Polygon(points); 
        
             return polygon.containsPoint(object[key]); 
        
           }

So i see they are different result of request between Javascript SDK and Flutter SDK
$maxDistance parameter working but $maxDistanceInKilometers and $geoWithin":"$centerSphere doesn't

When it comes to the Flutter SDK using $maxDistanceInKilometers, searching the reference files above shows that isn't supported by LiveQuery currently either.

You may want to open a PR to add support or edit your exiting PR.

cbaker6 · 2021-08-29T15:00:45Z

@mtrezza this should probably be labeled as an "enhancement" since it's known these operators aren't supported currently

mtrezza added the needs more info label Aug 4, 2021

PolSpock mentioned this issue Aug 27, 2021

Issue7480 max distance in kilometers params error #7526

Open

3 tasks

mtrezza added 🧬 enhancement and removed needs more info labels Aug 29, 2021

mtrezza added the bounty:$20 Bounty applies for fixing this issue (Parse Bounty Program) label Oct 7, 2021

mtrezza added type:feature New feature or improvement of existing feature and removed type:improvement labels Dec 6, 2021

This was referenced Oct 8, 2022

feat: liveQuery support for unsorted distance queries #8221

Merged

feat: localDatastore support for unsorted distance queries parse-community/Parse-SDK-JS#1570

Merged

mtrezza closed this as completed in #8221 Oct 11, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

withinKilometers withinMiles withinRadians not working with Live Query #7480

withinKilometers withinMiles withinRadians not working with Live Query #7480

PolSpock commented Aug 4, 2021 •

edited by mtrezza

Loading

mtrezza commented Aug 4, 2021

PolSpock commented Aug 4, 2021 •

edited

Loading

github-actions bot commented Aug 27, 2021 •

edited

Loading

mtrezza commented Aug 27, 2021 •

edited

Loading

cbaker6 commented Aug 29, 2021 •

edited

Loading

cbaker6 commented Aug 29, 2021 •

edited

Loading

withinKilometers withinMiles withinRadians not working with Live Query #7480

withinKilometers withinMiles withinRadians not working with Live Query #7480

Comments

PolSpock commented Aug 4, 2021 • edited by mtrezza Loading

New Issue Checklist

Issue Description

Steps to reproduce

Actual Outcome

Expected Outcome

Failing Test Case / Pull Request

Environment

Logs

mtrezza commented Aug 4, 2021

PolSpock commented Aug 4, 2021 • edited Loading

github-actions bot commented Aug 27, 2021 • edited Loading

🤖 Parsy

Thanks for opening this issue!

mtrezza commented Aug 27, 2021 • edited Loading

cbaker6 commented Aug 29, 2021 • edited Loading

cbaker6 commented Aug 29, 2021 • edited Loading

PolSpock commented Aug 4, 2021 •

edited by mtrezza

Loading

PolSpock commented Aug 4, 2021 •

edited

Loading

github-actions bot commented Aug 27, 2021 •

edited

Loading

mtrezza commented Aug 27, 2021 •

edited

Loading

cbaker6 commented Aug 29, 2021 •

edited

Loading

cbaker6 commented Aug 29, 2021 •

edited

Loading