Dev 5 1 2015 knn #1
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Created "GlobalIndex.java" class which implements "CatalogObject.java". - Implemented "CatalogObject.java" methods and added new methods for "GlobalIndex.java" class. - Created "GlobalIndexRecord.java" class to hold information about a single global index record. - Updated "CatalogObjects.thrift" to add a new catalog object type "GLOBAL_INDEX".
- Added a reference to "GlobalIndex.java" instance in "Table.java" catalog object class. - Added a method to get that instance and to call the loading of the GlobalIndex from HDFS file path.
- Updated method analyze() in "CreateTableStmt.java" to handle GI path. - Used "HdfsUri.java" to analyze the path.
- Created thrift structs: "TGlobalIndex", "TGlobalIndexRecord" and "TRectangle". - Handled the mapping between the java classes and the thrift structs. - Added methods "toThrift()" and "fromThrift()" to related java classes. - Added "TGlobalIndex" to "TTable" as an optional instance. - Handled transfroming "GlobalIndex" to thrift in "Table.java". - Handled transforming "TGlobalIndex" to "GlobalIndex" in "Table.java".
- Added method "loadGlobalIndex()" which loads the global index records provided the path of the file. - Handled returning loaded GlobalIndex to the Table. - Handled logging errors in case of a failure while parsing the file.
- Created class "SpatialHdfsTable.java" to hold GlobalIndex if the Hdfs table is Spatial. - Removed any reference in "Table.java" of "GlobalIndex". - Handled creating "SpatialHdfsTable.java" in case the table is Hdfs table and Spatial. - "SpatialHdfsTable" extends "HdfsTable" class and overrides methods "toThrift()" and "loadFromThrift()".
- Added logging while loading the global index file. - Handled returning null in case the global index has errors in SpatialHdfsTable while loading from thrift object.
- Added needed keywords to "sql-parser.y" and "sql-scanner.flex". - Created "SpatialPointInclusionStmt" to represent the new spatial query statement. - Handled parsing Rectangle as an object in "sql-parser.y". - Handled parsing the new spatial statement in "sql-parser.y". - Added "toString()" method to "Rectangle.java" to be used in "SpatialPointInclusionStmt".
- Handled analysis phase for "SpatialPointInclusionStmt". - Handled checking for table existence, table being spatial. - Handled checking for table containing GlobalIndex and containing required columns. - Updated "GlobalIndex" and "Rectangle" to return overlapping records with the provided Rectangle.
- Changed modifier of both "SelectStmt" and "SelectListItem" to be used inside the new statement. - Added getters to "x1, y1, x2 and y2" to be used inside the point inclusion statement. - Updated "SpatialPointInclusionStmt" to return the "SelectStmt" and it will be only available after analysis. - Handled creating "SelectStmt" providing the table reference, select list and where predicates. - Handled creating where predicate for "SelectStmt" with no global index overlapping. - Handled creating where predicate using the overlapping global indexes and the boundaries of the provided "Recatngle".
- Updated "AnalysisContext" to handle "SpatialPointInclusionStmt". - Added logging to "GlobalIndex" while checking for overlapping records. - Updated "overlaps()" method in "Rectangle" to be consistent with the "GlobalIndex" record. - Updated the where predicates in "SpatialPointInclusionStmt" to be consistent with the "GlobalIndex" records. - Tested the point inclusion statement on a partitioned spatial table. - Tested the statement in cases of rectangles out of the partitions, overlaps only one partition and overlaps two or more partitions.
1) Adding point class 2) Adding thrift point class 3) Adding KNN spatial stmt 4) Editting Global index 5) Editting Sql parser and scanner 6) Editting Analysis context
added 2 commits
January 5, 2015 19:39
Running Knn query with format: LOAD POINTS FROM TABLE [table_name] KNN [point(x,y)] WITH_K [k] WITH_DIST [distance equation/UDF]
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.