Positional parameters not detected inside SpEL expressions [DATACOUCH-616]

[spring-projects-issues]

Jonathan MASSUCHETTI opened DATACOUCH-616 and commented

@Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter} " +
 " #{#projectIds != null ? 'AND project IN $1' : ''} " +
 " #{#planIds != null ? 'AND ARRAY_LENGTH(ARRAY_INTERSECT(`plans`, $2)) > 0' : ''} " +
 " #{#active != null ? 'AND active = $3' : ''} " +
 " OFFSET $4 LIMIT $5")
 List<Phase> findAll(@Param("projectIds") List<String> projectIds, @Param("planIds") List<String> planIds, @Param("active") Boolean active, @Param("offset") Long offset, @Param("limit") Integer limit);

@Query("SELECT COUNT FROM #{#n1ql.bucket} WHERE #{#n1ql.filter} " +
 " #{#projectIds != null ? 'AND project IN $1 : ''} " +
 " #{#planIds != null ? 'AND ARRAY_LENGTH(ARRAY_INTERSECT(`plans`, $2)) > 0' : ''} " +
 " #{#active != null ? 'AND active = $3' : ''} ")
 Long count(@Param("projectIds") List<String> projectIds, @Param("planIds") List<String> planIds, @Param("active") Boolean active); 

 

The findAll query is correctly parsed, and receive positional parameters. OK
The count query is incorrectly parsed and crash before execution. KO

 

Error is the following :

Found non terminating string literal starting at position 252\n\tat org.springframework.expression.common.TemplateAwareExpressionParser.skipToCorrectEndSuffix(TemplateAwareExpressionParser.java:208)

 

---

Affects: 3.2.10 (Moore SR10)

Referenced from: pull request #265

Backported to: 4.0.5 (Neumann SR5)

[spring-projects-issues]

Michael Reiche commented

Query string is missing a quote after the $1

 

" #{#projectIds != null ? 'AND project IN $1 : ''} " +

[spring-projects-issues]

Michael Reiche commented

There is a missing quote in the query

[spring-projects-issues]

Jonathan MASSUCHETTI commented

Should have gone to bed earlier... sry for the silly issue

[spring-projects-issues]

Jonathan MASSUCHETTI commented

I had spent the evening trying to figure things out. The quote mistake was silly, but it is not all of the story.

I've updated the query to this : 

@Query("SELECT COUNT(*) FROM #{#n1ql.bucket} WHERE #{#n1ql.filter} " +
    " #{#projectIds != null ? 'AND project IN $1' : ''} " +
    " #{#planIds != null ? 'AND ARRAY_LENGTH(ARRAY_INTERSECT(`plans`, $2)) > 0' : ''} " +
    " #{#active != null ? 'AND active = $3' : ''} ")
Long count(@Param("projectIds") List<String> projectIds, @Param("planIds") List<String> planIds, @Param("active") Boolean active);

AbstractN1qlBasedQuery logs the resulting query as : 

SELECT COUNT(*) FROM `myBucket` WHERE `_class` = "myClass" AND project IN $1

CouchbaseQueryExecutionException then throw this error :

Error evaluating span. - cause: No value for positional parameter $1

PS : Michael Reiche I apologize for the messiness of this ticket

[spring-projects-issues]

Michael Reiche commented

JesusTheHun I'm looking into it

[spring-projects-issues]

Michael Reiche commented

JesusTheHun - the query strings were being parsed for parameters before the conditionals were being resolved - the portions containing the query parameters were still enclosed in quotes and therefore not being  recognized as query parameters.

Also - for your "count" query - also begin with the same query you have for the "find" query.  For spring-data-couchbase, count queries are recognized by method names beginning with "count", and the appropriate "SELECT COUNT( * ) count FROM  ..."  This insures the result set returns the correctly named element. 

@Query("#{#n1ql.selectEntity} WHERE #{#n1ql.filter} " +

If the result does not contain "count", an NPE is thrown indicating the mapper returned a null value

java.lang.NullPointerException: The mapper returned a null value.

And curiously, if an unrecognized project name is specified, and there are no matching documents, a mysterious NullPointerException is thrown from the Flux block() call.

   @Query("SELECT junk_qwerty FROM #{#n1ql.bucket} WHERE #{#n1ql.filter} " +
  " #{#projectIds != null ? 'AND blah IN $1' : ''} " +
  " #{#planIds != null ? 'AND blahblah IN $2' : ''} " +
  " #{#active != null ? 'AND false = $3' : ''} ")
Long count(@Param("projectIds") List<String> projectIds, @Param("planIds") List<String> planIds, @Param("active") Boolean active);

 

java.lang.NullPointerExceptionjava.lang.NullPointerException at org.springframework.data.couchbase.core.ExecutableFindByQueryOperationSupport$ExecutableFindByQuerySupport.count(ExecutableFindByQueryOperationSupport.java:90) at org.springframework.data.couchbase.repository.query.N1qlRepositoryQueryExecutor.execute(N1qlRepositoryQueryExecutor.java:70) at org.springframework.data.couchbase.repository.query.CouchbaseRepositoryQuery.execute(CouchbaseRepositoryQuery.java:42)

 

[spring-projects-issues]

Jonathan MASSUCHETTI commented

See :

spring-data-couchbase/src/main/java/org/springframework/data/couchbase/repository/query/AbstractN1qlBasedQuery.java

Line 119 in 946e835

JsonValue queryPlaceholderValues = getPlaceholderValues(accessor);

This is the master branch but it's the same order in the 3.x branch.
The statement contain the spel parsed string and it's correct.
This issue seems to be the variable StringBasedN1qlQueryParser.placeholderType not being set (so defaulting to NONE) therefore no parameter matching occurs.

I thought about the method name too, so I tried with foo(...), same issue.

In your last case if there are documents an exception will be thrown saying query returning a primitive expect only one document. So an NPE is not the right error but an error is expected anyway

[spring-projects-issues]

Jonathan MASSUCHETTI commented

Ok I see what you meant now, the detection of the placeholder types occurs before the spel expression being parsed, so it's not detected, hence placeholderType being NONE .
For obvious performance reasons it's better to parse it once and for all so I guess there will be not fix for that ?
The reason the find query works is because there are positional parameters out of quotes

[spring-projects-issues]

Jonathan MASSUCHETTI commented

Hello Michael Reiche, I see the PR has been closed, did you drop the idea of fixing it ?

[spring-projects-issues]

Michael Reiche commented

JesusTheHun - the commit has been cherry-picked into master

[spring-projects-issues]

Michael Reiche commented

fixed in 4.0.5 and 4.1.0.
Need to backport to 3.2.x

[spring-projects-issues]

Mark Paluch commented

Please create a new ticket for the backport as we're releasing today