Introduce Search API

[stephenplusplus]

Fixes #632

Docs: https://stephenplusplus.github.io/gcloud-node/#/docs/master/search

To dos:

• Enable stream mode in Search#getIndexes()
• Enable stream mode in Index#getDocuments()
• Enable stream mode in Index#search()
• Docs
  • JSDocs
  • Readme
  • Site
    • Update sitemap
    • Add "Search Overview" section
  • StreamRouter
• Tests
  • Unit
  • System

API can't-do's:

• Can't delete an index (can list indexes and delete all documents)
• Can't create an index directly (only when creating a document)
• Can't search a specific document

Not perfected yet:

• Document JSON (when creating a document, we may be able to ease the field specification process)
  • See example below
• Queries (we can probably have an object builder rather than just accepting a string)
  • It seems better to allow the developer to use the familiar query language as outlined in the docs. Our re-implementing the grammar could introduce inconsistencies and prevent portability.

Examples:

Create an index / Create a document from scratch

var search = gcloud.search();
var index = search.index('gcloud-services');

var document = index.document('search');
document.addField('implemented').addAtomValue('almost');

index.createDocument(document, function(err, document, apiResponse) {});

Create a document from an existing JSON object / file

var search = gcloud.search();
var index = search.index('gcloud-services');

var documentJson = {
  docId: 'search',
  fields: {
    values: [
      implemented: {
        stringValue: 'almost',
        stringFormat: 'ATOM'
      }
    ]
  }
};

index.createDocument(documentJson, function(err, document, apiResponse) {});

Get search results with a callback

var search = gcloud.search();
var index = search.index('gcloud-services');

function onSearchResults(err, documents, nextQuery, apiResponse) {
  if (err) {
    console.error(err);
    return;
  }

  // documents[0].id = 'search';

  if (nextQuery) {
    index.search(nextQuery, onSearchResults);
  }
}

index.search('implemented:almost', onSearchResults);

Stream search results

var search = gcloud.search();
var index = search.index('gcloud-services');

index.search('implemented:almost')
  .on('error', console.error)
  .on('data', function(document) {
    // document.id = 'search';
  });

[stephenplusplus]

Index#getDocuments and Index#search are now stream-able.

Also, I added document building functionality and updated the first post here with some examples.

[stephenplusplus]

The todo list is a-ticking away. Now would be a good time to catch any mis-steps (see the docs). Next to tackle are the tests.

[stephenplusplus]

This is ready for review (from who, I don't know 😄). Todos completed.

[ryanseys]

I will
On Jun 17, 2015 6:13 AM, "Stephen Sawchuk" [email protected] wrote:

This is ready for review (from who, I don't know [image: 😄]). Todos
completed.

—
Reply to this email directly or view it on GitHub
#648 (comment)
.

[ryanseys]

I run npm test and get the following console logs at the end of the test results (no failures):

{ [Error: ENOENT, open '/path/to/keyfile.json'] errno: -2, code: 'ENOENT', path: '/path/to/keyfile.json' }
{ [Error: ENOENT, open '/path/to/keyfile.json'] errno: -2, code: 'ENOENT', path: '/path/to/keyfile.json' }
{ [Error: ENOENT, open '/path/to/keyfile.json'] errno: -2, code: 'ENOENT', path: '/path/to/keyfile.json' }

[stephenplusplus]

I thought that might just be me. Can you advise how to debug that?

[ryanseys]

I think it's because we pass the external console object into the vm object when we test our code docs, so if the docs that are tested have any calls to console.log or console.error then they will be written to our external stdout. This is the case with some of your tests when you run .on('error', console.error).

Easiest way to fix is overwrite console.log and console.error for the docs test.

[ryanseys]

Looks GREAT to me!

[stephenplusplus]

Wee! I just added this in to handle pre-mature closing of a stream: stephenplusplus@abca6f1

Basically to handle something like this:

index.getDocuments().on('data', function(document) {
  if (document.id === 'the-one-i-want') {
    this.end();
  }
});

Before the change, we would have had no idea end was called, and would have kept trying to push documents to the stream & even continue calling the API for more results.

[stephenplusplus]

Merge-time!