Discussing adding separate system tests for REST and gRPC.

[daspecster]

Per what we talked about a little bit today.

It would probably be a good idea to run both the REST and gRPC code in the system tests.
I think most services that could have both REST and gRCP tests could use skipTest to manage when it is appropriate to run them.(#2648 (comment))

When to run them?

I think @dhermes brought up that we shouldn't run both of them all the time and that maybe we should use Travis-CI cron to run them once or twice a day. We could also just cron building master 1-4 times per 24 hours instead of on every merge.

[tseaver]

The Travis cron jobs docs say:

Cron jobs can run daily, weekly or monthly, which in practice means up to an hour after the selected time span, and they can also be skipped. Cron jobs cannot be set to run at specific times.

They also say:

Please note that cron jobs are not enabled by default. Set “Build pushes” to on in your settings, then ask us to unlock this feature for your repository: support@travis-ci.com.

Detecting a cron-triggered build:

To check whether a build was triggered by cron, examine the TRAVIS_EVENT_TYPE environment variable to see if it has the value cron.

[daspecster]

@tseaver I think we could make 4 cron jobs to get it to run 4 times a day. Just change the time span of when each one can run in.

[tseaver]

@daspecster Looking at the dialog for adding a cron job. I don't see any knob for tweaking time-of-day.

[dhermes]

Why would we need to run more than once a day?

[daspecster]

@tseaver maybe not, I'm not sure what's in the options drop down there. The way it's worded makes it sound like you may be able to select some portion of the day.

@dhermes not sure that we do, but it would be nice to have it run at night or early in the morning.

[dhermes]

@lukesneeringer This is actually important.

[lukesneeringer]

I actually thought this was done.

[dhermes]

@lukesneeringer ISTM that the "correct" way to proceed is to

• Do both HTTP and gRPC in nightlies
• One of
  • Never do HTTP in "push" builds
  • Randomly pick one of HTTP / gRPC in "push" builds based on e.g. build_number % 2