Sampling texture with nearest interpolation

[justanhduc]

🚀 Feature

Currently, texture sampling is done with bilinear interpolation. It's great if there is an option to choose which interpolation to be done too.

Motivation

In some case, for e.g. when the texture image is segmentation labels, users may prefer nearest to prevent class labels from being interpolated into nonsense values.

Pitch

textures = TexturesUV(texture_img, face_uvs, verts_uv, interp_mode='nearest')

I can make a PR if this request is approved. Thanks!

[bottler]

This is a bit similar to issue #798. It means you have one label per vertex, and a triangle face can get split, in general, into three quadrilaterals or a quadrilateral and a pentagon. But instead of storing the labels per vertex, you store them in a texture map. Just like it was easy to approach that with a custom subclass of TexturesVertex, you could solve this with a custom subclass of TexturesUV. But it would be slightly easier if TexturesUV exposed the option itself.

I'd be interested to understand the application better though. Do you (or users you imagine) have data which is actually like this: a segment map together with faces whose vertices are in separate segments which you want to be divided? Or does this just become a problem after some optimization steps (e.g. because you are sharing learned UV coordinates between two TexturesUV, one of which represents segments and one of which is colors)? Do you ideally want the segment to be defined on faces or on vertices?

[justanhduc]

Hi @bottler. Yeah actually it is a problem i'm facing now. I need to estimate a texture image given multiple views (and their corresponding seg maps) and the mesh vt and ft. So what I do is I use the vt and ft as a proxy mesh, and the projected mesh on a 2D view as uv, and then I render the proxy mesh which will give me the estimated texture image from each view. For the next steps, I need to do the same for the seg maps as well, which led me to this issue. I solved this simply by using 'nearest' for grid_sample, and I think it would be nicer to expose this as an interface (if it is applicable to other users as well) rather than have to subclass the whole thing for just a one-line change like this.

[nikhilaravi]

@justanhduc do you also need to apply lighting? The lighting code will use the barycentric coordinates to apply lighting so you may also need to update this or use a custom shader without lights as explained in #806.

[justanhduc]

Hi @nikhilaravi. In my case lighting is not needed because I need the true colors which represent the labels. I simply rendered without any shader.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[nikhilaravi]

@justanhduc do you still need help with this question?

[justanhduc]

Hi @nikhilaravi. Thanks I actually solved the problem and was asking a possible PR if there is interest. I will close the issue here.