Skip to content

cuda.core: Change selected .decode() calls to .decode("utf-8", errors="backslashreplace") #510

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 12, 2025

Conversation

rwgk
Copy link
Collaborator

@rwgk rwgk commented Mar 12, 2025

Change selected .decode() calls to .decode("utf-8", errors="backslashreplace"), to avoid UnicodeDecodeError exceptions when converting error messages or program logs.

Fixes this error (when testing under Windows):

if _nvjitlink:
    log_size = _nvjitlink.get_error_log_size(self._mnff.handle)
    log = bytearray(log_size)
    _nvjitlink.get_error_log(self._mnff.handle, log)
else:
    log = self._options.formatted_options[2]
>   return log.decode()
E UnicodeDecodeError: 'utf-8' codec can't decode byte 0xaa in position 47: invalid start byte

Only the two changes in _linker.py are covered by existing unit tests, and only with valid utf-8 strings. I.e. the conditions for which this PR makes a difference are not actually exercised. — Making the test coverage more complete is left for later.

The selection in based on a manual review of all .decode() calls under cuda_core:

$ git grep 'decode('
cuda/core/experimental/_device.py:        return bus_id[:12].decode()
cuda/core/experimental/_device.py:        return name.decode()
cuda/core/experimental/_linker.py:        return log.decode("utf-8", errors="backslashreplace")
cuda/core/experimental/_linker.py:        return log.decode("utf-8", errors="backslashreplace")
cuda/core/experimental/_program.py:                    logs.write(log.decode("utf-8", errors="backslashreplace"))
cuda/core/experimental/_utils/cuda_utils.py:            raise CUDAError(f"{name.decode()}: {desc.decode()}")
cuda/core/experimental/_utils/cuda_utils.py:            raise CUDAError(f"{name.decode()}: {desc.decode()}")
cuda/core/experimental/_utils/cuda_utils.py:        err = f"{error}: {nvrtc.nvrtcGetErrorString(error)[1].decode()}"
cuda/core/experimental/_utils/cuda_utils.py:            err += f", compilation log:\n\n{log.decode('utf-8', errors='backslashreplace')}"
tests/test_device.py:    assert device.pci_bus_id == bus_id[:12].decode()
tests/test_device.py:    assert device.name == name.decode()
tests/test_program.py:    program = Program(ptx_code_object._module.decode(), "ptx", options=options)
tests/test_program.py:    program = Program(ptx_object_code._module.decode(), "ptx")

Copy link
Contributor

copy-pr-bot bot commented Mar 12, 2025

Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@rwgk
Copy link
Collaborator Author

rwgk commented Mar 12, 2025

/ok to test

@rwgk rwgk requested review from leofang and ksimpson-work March 12, 2025 06:52

This comment has been minimized.

@rwgk
Copy link
Collaborator Author

rwgk commented Mar 12, 2025

Thanks @ksimpson-work, I'll merge this now, to support QA testing.

@rwgk rwgk merged commit ba8de1d into NVIDIA:main Mar 12, 2025
73 checks passed
@rwgk rwgk deleted the decode_backslashreplace branch March 12, 2025 18:42
Copy link

Doc Preview CI
Preview removed because the pull request was closed or merged.

@leofang leofang added bug Something isn't working P0 High priority - Must do! cuda.core Everything related to the cuda.core module labels Mar 22, 2025
@leofang leofang added this to the cuda.core beta 3 milestone Mar 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cuda.core Everything related to the cuda.core module P0 High priority - Must do!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants