Skip to content

Conversation

@chriselion
Copy link
Contributor

@chriselion chriselion commented Sep 16, 2020

Proposed change(s)

We currently get warnings when stopping training in the GridFoodCollector scene, because the AcademyFixedUpdateStepper is recreated. This happens because in Agent.OnDisable, we try to send the observations to the communicator (which has been shut down already). For the case of GridFoodCollector, this ends up calling Academy.Instance.TrainerCapabilities here

var trainerCanHandle = Academy.Instance.TrainerCapabilities == null || Academy.Instance.TrainerCapabilities.ConcatenatedPngObservations;

which recreates the Academy instance.

There are some narrower fixes, like checking Academy.IsInitialized in GetObservationProto, or make RpcCommunicator early out when it has been shutdown (checking m_IsOpen). But in general it feels like a bug if the Agent is trying to do any sort of "work" when everything is shut down.

I'm currently targeting the release branch but I'd be willing to wait and make it on master instead if you think it's too risky/too late in the process.

Useful links (Github issues, JIRA tickets, ML-Agents forum threads etc.)

Types of change(s)

  • Bug fix

Checklist

  • Added tests that prove my fix is effective or that my feature works
  • Updated the changelog (if applicable)
  • Updated the documentation (if applicable)
  • Updated the migration guide (if applicable)

Other comments

@chriselion chriselion merged commit 2a69c1f into release_7_branch Sep 16, 2020
@delete-merged-branch delete-merged-branch bot deleted the r7-dont-notify-if-academy-off branch September 16, 2020 17:03
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 16, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants