Visual observations for internal brain are not normalized properly

[fre]

Had an issue with my internal brain behaving differently than the external brain in inference mode when passed visual observations (a hand-generated 16x16 texture).

It seems that images received by environment.py are normalized to [0;1] (pixel values are divided by 255):

ml-agents/python/unityagents/environment.py

Line 196 in 8365e2c

s = np.array(image) / 255.0

But the values extracted by CoreBrainInternal::BatchVisualObservations are not:

ml-agents/unity-environment/Assets/ML-Agents/Scripts/CoreBrainInternal.cs

Line 528 in 8365e2c

result[b, textures[b].height - h - 1, w, 0] = currentPixel.r;

which leads to unpredictable agent behavior (since inputs are 255x bigger than expected).

This solves my issue:

{
    result[b, textures[b].height - h - 1, w, 0] = currentPixel.r / 255.0f;
    result[b, textures[b].height - h - 1, w, 1] = currentPixel.g / 255.0f;
    result[b, textures[b].height - h - 1, w, 2] = currentPixel.b / 255.0f;
}
else
{
    result[b, textures[b].height - h - 1, w, 0] = (currentPixel.r + currentPixel.g + currentPixel.b) / 3  / 255.0f;
}```

[vincentpierre]

Hi, Color.r, Color.g and Color.b return floats between 0 and 1. Source here We divide on python because we send the image as PNG instead of a Texture2D. Do you manage to have the gridworld working ? It surprises me that you get better results by dividing by 255. Could you compare the performance for gridworld?

[fre]

This code is using Color32, which returns values between 0 and 255.
I've only tested this on my own experimental project where I use visual observations to pass the state of the game (think chessboard state) rather than camera renders, so it may be much more sensitive to scaling than the example.

[superjayman]

So, is this a bug?

[vincentpierre]

I think you are right, this is a bug. I will try to get is fixed. Thank you for your help @fre.

[fre]

AFAIK, this bug only affects internal brains, i.e. importing the trained agent in unity. Training a gridworld-type task works fine for me. FYI I generate a 16x16 board representation centered on the agent where each type of entity is rendered on a different channel (R,G,B) of the output texture and only use a single 3x3 convolution layer with 256 filters and stride 1 before a single fully connected layer of size 128. Slowing down the time scale during training (adding UI buttons for that is helpful) shows that the agent learns properly. It's only when I import the trained model in unity that the internal brain behaves differently (without this normalization fix).

…

On Sat, Apr 14, 2018, 9:46 AM Marco Pleines ***@***.***> wrote: I'm wondering if this is why I could not get anything environment working, which relies on visuals only. (#198 <#198>) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#614 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAFakKdl9TGPHGSqsYZmN_zuTAJGJzlxks5toalwgaJpZM4TQdn3> .

[RossMelbourne]

Thank you very much fre for this fix. I applied fre's code above and it fixed a very nagging problem I have been having. Now my app work just as well in Internal mode!

[xiaomaogy]

Hi @fre @RossMelbourne, we are closing this issue due to inactivity, feel free to open it if you need more help on this, hopefully Vince's fix solves your issue.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.