CPT range auto-stretching works differently with NetCDF and virtual reliefgrid inputs

[weiji14]

Description of the problem

When comparing a grdimage plot from a regular remote file and virtual file input, while setting a specific region (e.g. Greenland/GL in the case below), the CPT range is stretched differently. Specifically, the CPT range is -3893 to 3195.5 for the remote file case, but -8592.5 to 5559 for the virtual file case. See GenericMappingTools/pygmt#750 (comment) for context.

expected (correct)	actual (wrong)	diff

Need to install PyGMT branch from GenericMappingTools/pygmt#750 first using pip install https://github.com/GenericMappingTools/pygmt/archive/grdimage-xarray-shading.zip first before running the below: Edit: just do pip install pygmt=0.5.0.

Full script that generated the error

import numpy as np
import pygmt
import xarray as xr

# xarray.DataArray grid used as input for shading (-I) parameter
longitude = np.arange(0, 360, 1)
latitude = np.arange(-89, 90, 1)
x = np.sin(np.deg2rad(longitude))
y = np.linspace(start=0, stop=1, num=179)
data = y[:, np.newaxis] * x

xrgrid = xr.DataArray(
    data,
    coords=[
        ("latitude", latitude, {"units": "degrees_north"}),
        ("longitude", longitude, {"units": "degrees_east"}),
    ],
    attrs={"actual_range": [-1, 1]},
)

# Baseline (correct) case using @earth_relief_01d_g reliefgrid
fig = pygmt.Figure()
fig.grdimage(
    grid="@earth_relief_01d_g", region="GL", cmap="geo", shading=xrgrid, verbose="i"
)
fig.colorbar()
fig.show()


# Expected (wrong) case using xarray.DataArray reliefgrid
fig = pygmt.Figure()
grid = pygmt.datasets.load_earth_relief(registration="gridline")
fig.grdimage(grid=grid, region="GL", cmap="geo", shading=xrgrid, verbose="i")
fig.colorbar()
fig.show()

Full error message

Copied from https://github.com/GenericMappingTools/pygmt/runs/2733822414?check_suite_focus=true#step:11:543

Actual outcome

Shading an xarray.DataArray (via the GMT C API)

**

grdimage [INFORMATION]: Using country and state data from dcw-gmt
grdimage [INFORMATION]: Title  : DCW-GMT - The Digital Chart of the World for the Generic Mapping Tools
grdimage [INFORMATION]: Source : Processed by the GMT Team, 2018-JUL-01
grdimage [INFORMATION]: Version: 1.1.4
grdimage [INFORMATION]: Greenland
grdimage [INFORMATION]: Region implied by DCW polygons is 286.737/348.688/59.7773/83.6274
grdimage [INFORMATION]: Read intensity grid header from file @GMTAPI@-S-I-G-M-G-N-000000
grdimage [INFORMATION]: Read header from file @GMTAPI@-S-I-G-M-G-N-000001
grdimage [INFORMATION]: Spherical approximation used
grdimage [INFORMATION]: Central meridian not given, default to 317.712
grdimage [INFORMATION]: Map scale is 459.244 km per cm or 1:4.59244e+07.
grdimage [INFORMATION]: Allocate and read data from file @GMTAPI@-S-I-G-M-G-N-000001
grdimage [INFORMATION]: Importing grid data from user matrix memory location
grdimage [INFORMATION]: called with a pad < 2; skipped.
grdimage [INFORMATION]: Allocates memory and read intensity file
grdimage [INFORMATION]: Importing grid data from user matrix memory location
grdimage [INFORMATION]: Set boundary condition for all edges: natural
grdimage [INFORMATION]: Set boundary condition for left   edge: natural
grdimage [INFORMATION]: Set boundary condition for right  edge: natural
grdimage [INFORMATION]: Set boundary condition for bottom edge: natural
grdimage [INFORMATION]: Set boundary condition for top    edge: natural
grdimage [INFORMATION]: Reading CPT from File /usr/share/miniconda3/envs/pygmt/share/gmt/cpt/geo.cpt
grdimage [INFORMATION]: Auto-stretching CPT file geo to fit data range -8592.5 to 5559
grdimage [INFORMATION]: Write CPT to File /home/runner/.gmt/sessions/gmt_session.5828/gmt.78.cpt
grdimage [INFORMATION]: Save current CPT file to /home/runner/.gmt/sessions/gmt_session.5828/gmt.78.cpt !
grdimage [INFORMATION]: Evaluate image pixel colors
grdimage [INFORMATION]: Basic z(x,y) -> color image with no illumination.
grdimage [INFORMATION]: Plotting 24-bit color image
PSL: Too many colors to make colormap - using 24-bit direct color instead.
PSL: DEFLATE compressed 4725 to 4231 bytes (10.5% savings at compression level 5)

Expected outcome

Shading the @earth_relief_01d_g file directly.

grdimage [INFORMATION]: Using country and state data from dcw-gmt
grdimage [INFORMATION]: Title  : DCW-GMT - The Digital Chart of the World for the Generic Mapping Tools
grdimage [INFORMATION]: Source : Processed by the GMT Team, 2018-JUL-01
grdimage [INFORMATION]: Version: 1.1.4
grdimage [INFORMATION]: Greenland
grdimage [INFORMATION]: Region implied by DCW polygons is 286.737/348.688/59.7773/83.6274
grdimage [INFORMATION]: Read intensity grid header from file @GMTAPI@-S-I-G-M-G-N-000000
grdimage [INFORMATION]: Read header from file /home/runner/.gmt/server/earth/earth_relief/earth_relief_01d_g.grd
grdimage [INFORMATION]: Spherical approximation used
grdimage [INFORMATION]: Central meridian not given, default to 317.712
grdimage [INFORMATION]: Map scale is 459.244 km per cm or 1:4.59244e+07.
grdimage [INFORMATION]: Allocate and read data from file /home/runner/.gmt/server/earth/earth_relief/earth_relief_01d_g.grd
grdimage [INFORMATION]: Reading grid from file /home/runner/.gmt/server/earth/earth_relief/earth_relief_01d_g.grd
grdimage [INFORMATION]: All boundaries set via extended data.
grdimage [INFORMATION]: Allocates memory and read intensity file
grdimage [INFORMATION]: Importing grid data from user matrix memory location
grdimage [INFORMATION]: Set boundary condition for all edges: natural
grdimage [INFORMATION]: Set boundary condition for left   edge: natural
grdimage [INFORMATION]: Set boundary condition for right  edge: natural
grdimage [INFORMATION]: Set boundary condition for bottom edge: natural
grdimage [INFORMATION]: Set boundary condition for top    edge: natural
grdimage [INFORMATION]: Reading CPT from File /usr/share/miniconda3/envs/pygmt/share/gmt/cpt/geo.cpt
grdimage [INFORMATION]: Auto-stretching CPT file geo to fit data range -3893 to 3195.5
grdimage [INFORMATION]: Write CPT to File /home/runner/.gmt/sessions/gmt_session.5828/gmt.77.cpt
grdimage [INFORMATION]: Save current CPT file to /home/runner/.gmt/sessions/gmt_session.5828/gmt.77.cpt !
grdimage [INFORMATION]: Evaluate image pixel colors
grdimage [INFORMATION]: Basic z(x,y) -> color image with no illumination.
grdimage [INFORMATION]: Plotting 24-bit color image
PSL: Too many colors to make colormap - using 24-bit direct color instead.
PSL: DEFLATE compressed 4725 to 4375 bytes (7.4% savings at compression level 5)

Diff

-grdimage [INFORMATION]: Allocate and read data from file /home/runner/.gmt/server/earth/earth_relief/earth_relief_01d_g.grd	
-grdimage [INFORMATION]: Reading grid from file /home/runner/.gmt/server/earth/earth_relief/earth_relief_01d_g.grd	
-grdimage [INFORMATION]: All boundaries set via extended data.
+grdimage [INFORMATION]: Allocate and read data from file @GMTAPI@-S-I-G-M-G-N-000001
+grdimage [INFORMATION]: Importing grid data from user matrix memory location
+grdimage [INFORMATION]: called with a pad < 2; skipped.

-grdimage [INFORMATION]: Auto-stretching CPT file geo to fit data range -3893 to 3195.5	
-grdimage [INFORMATION]: Write CPT to File /home/runner/.gmt/sessions/gmt_session.5828/gmt.77.cpt	
-grdimage [INFORMATION]: Save current CPT file to /home/runner/.gmt/sessions/gmt_session.5828/gmt.77.cpt !
+grdimage [INFORMATION]: Auto-stretching CPT file geo to fit data range -8592.5 to 5559
+grdimage [INFORMATION]: Write CPT to File /home/runner/.gmt/sessions/gmt_session.5828/gmt.78.cpt
+grdimage [INFORMATION]: Save current CPT file to /home/runner/.gmt/sessions/gmt_session.5828/gmt.78.cpt !

System information

• Operating system: Linux
• GMT version (gmt --version): 6.2.0rc2

[PaulWessel]

I am getting lots of these errors (the one with -1 -9 etc):

...
grdimage [ERROR]: Unrecognized option -1
grdimage [ERROR]: Unrecognized option -9
grdmix [ERROR]: Cannot find file (longitude:
grdmix [ERROR]: Cannot find file 360)>
grdimage [ERROR]: Unable to combine @GMTAPI@-S-I-G-M-G-N-000000/(longitude:/360)> into an image - aborting.
grdimage [ERROR]: Option -I: Must specify intensity file, value, or modifiers

What am I missing here?

[weiji14]

Ah yes, could you try doing pip install https://github.com/GenericMappingTools/pygmt/archive/grdimage-xarray-shading.zip in the terminal first. The shading feature only works on @seisman's PR branch and not on PyGMT master.

[PaulWessel]

Sorry, I need more help if I am going to debug this - we are also running out on time fro 6.2.0 I think.

pip install github.com/GenericMappingTools/pygmt/archive/grdimage-xarray-shading.zip
WARNING: Requirement 'github.com/GenericMappingTools/pygmt/archive/grdimage-xarray-shading.zip' looks like a filename, but the file does not exist

I usually debug against master and now little about the pyGMT setup...

[weiji14]

Hmm, did you add https:// to the url?

[PaulWessel]

For some reason I managed not to copy that part... OK now.

[PaulWessel]

But makes no difference - same errors.

[weiji14]

Did you restart the Python kernel? I'm assuming you ran pip install after conda activate pygmt. Perhaps take a step back following https://github.com/GenericMappingTools/gmt/blob/master/doc/rst/source/debug.rst#debug-pygmt-in-xcode-on-macos:

conda activate pygmt  # Activate the PyGMT environment
pip install https://github.com/GenericMappingTools/pygmt/archive/grdimage-xarray-shading.zip  # install the PyGMT PR branch
python  # to get the python console
ipython  # to get interactive python console

[PaulWessel]

I probably need to install ipython then

ipython
Traceback (most recent call last):
File "", line 1, in
NameError: name 'ipython' is not defined

[PaulWessel]

sudo port install ipython
Password:
Error: Port ipython not found

[maxrjones]

conda install ipython

[PaulWessel]

OK, done, but still typing ipython into the python problem gives same error.

[weiji14]

Sorry, just ignore the ipython command. That's an alternative to python...

[PaulWessel]

OK, getting same errors as reported at first.
is the alleged bug not reproducible with master at all?