Health API Showing OK when unable to connect to database

[toddheslin]

Issue Description

I've had a recent issue where I was relying on pingdom hitting my /health endpoint expecting that it should show an error if there is a database connection error. Unfortunately this is not the case. The Parse app connects to a local postgres database that crashed and didn't restart. Whilst the application was technically crashed, the health endpoint responded OK.

I've since updated to the latest release and see this is still a problem. Is this intentional, or just overlooked?

Steps to reproduce

1. Start server
2. GET /parse/health -> response {"status":"ok"}
3. Switch off database
4. Error shows in logs:

{ Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
  code: 'ECONNREFUSED',
  errno: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 5432 }
warn: Unable to ensure uniqueness for usernames:  Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
warn: Unable to ensure uniqueness for user email addresses:  Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
warn: Unable to ensure uniqueness for role name:  Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
(node:8526) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 8): Error: connect ECONNREFUSED 127.0.0.1:5432
(node:8526) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:8526) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 9): Error: connect ECONNREFUSED 127.0.0.1:5432

5. Check health endpoint: GET /parse/health -> response {"status":"ok"}

Expected Results

Health status should not be ok if there is a database connection error

Actual Outcome

Health status returning ok despite the application not able to make receive requests

Environment Setup

• Server

  • parse-server version (Be specific! Don't say 'latest'.) : 2.7.2
  • Operating System: macOS 10.13.3
  • Hardware: MacBook Pro
  • Localhost or remote server? (AWS, Heroku, Azure, Digital Ocean, etc): Localhost

• Database

  • Postgres version: 9.6.6
  • Storage engine: N/A
  • Hardware: N/A
  • Localhost or remote server? (AWS, mLab, ObjectRocket, Digital Ocean, etc): Localhost

Logs/Trace

{ Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
  code: 'ECONNREFUSED',
  errno: 'ECONNREFUSED',
  syscall: 'connect',
  address: '127.0.0.1',
  port: 5432 }
warn: Unable to ensure uniqueness for usernames:  Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
warn: Unable to ensure uniqueness for user email addresses:  Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
warn: Unable to ensure uniqueness for role name:  Error: connect ECONNREFUSED 127.0.0.1:5432
    at Object._errnoException (util.js:1022:11)
    at _exceptionWithHostPort (util.js:1044:20)
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1182:14)
(node:8899) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 8): Error: connect ECONNREFUSED 127.0.0.1:5432
(node:8899) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:8899) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 9): Error: connect ECONNREFUSED 127.0.0.1:5432
GET /api/health 200 4.825 ms - 15

[montymxb]

@toddheslin currently the health endpoint is a simple way for checking the status of parse-server itself, rather than having to make a test query or something like that. As such it doesn't provide any details in terms of the database connection.

I actually brought this up some time ago in #4305, but there were a number of concerns about making too much information available from a publicly accessible endpoint. We ended up settling for a structured JSON response with the existing OK value as the status and nothing more. If I remember correctly this starts to get a tad confusing when you have multiple parse server instances behind a load balancer. Still, it is something that could be nice to have in the future, provided it is not public and still useful when running more than one instance of parse server.

[toddheslin]

thanks @montymxb.

One solution I thought of could be setting up Pingdom to make a POST to the Parse Server api endpoint. E.g. if my Parse Server is hosted at /api, I could use Pingdom to ping:
POST {url}/api/Health with an empty (or not) body and an API key in the header.

This would be loading a new Parse object of type Health in the database. Could keep these as a database log of if/when downtime occurred, or simply clean it up using an afterSave cloud function hook.

I haven't actually implemented this yet, but it seems a simple way to ensure everything works.

With respect to sending back other information about the server, I guess this could be done by calling the cloud function with the master key. The function could return whatever you wanted to know about the server. Once again, haven't tried this but it seems reasonably possible.

[montymxb]

Np. I haven't experimented returning server related information from cloud code, but it could be possible. Might be a way to set up something for your needs beyond the standard health endpoint.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.