SDK tests must validate pub get/upgrade

[kevmoo]

RE #32655 snafu

[dgrove]

This isn't a P0, but it is something we need to address.

[matanlurey]

I opened a similar bug (and got it closed) a while ago.

I realize we are "moving fast and breaking things", but I think at some point we need to have release verification, not just an automatic process of pushing releases without end to end testing for core functionality.

[dgrove]

We will fix this.

…

On Fri, Mar 23, 2018 at 9:54 AM Matan Lurey ***@***.***> wrote: I opened a similar bug (and got it closed) a while ago. I realize we are "moving fast and breaking things", but I think at some point we need to have release verification, not just an automatic process of pushing releases without end to end testing for core functionality. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#32656 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACAsW7b5FvWfDjITtd4trT8UuZEalfbOks5thSjNgaJpZM4S5DP9> .

[athomas]

Breakdown after brief chat with @kevmoo:

• Migrate pkg builders to the test matrix
• Add step(s) to pub get/pub upgrade package:shelf or test

[athomas]

Status update:

• Moved the pkg builders to the test matrix, that makes it easier for me to proceed.
• Built a prototype for the pkg integration test: works but is a bit hackish.
• Running pub is required to run scripts to release the SDK, when I built 2.0.0.49-dev I used the pub from the SDK to publish it. I will keep doing that until there is a test for this.