Enable string conversion in EUC-JP. #1296
Conversation
|
@swift-ci Please test |
| // Attempt an up-call into swift-corelibs-foundation, which can defer to the CoreFoundation implementation | ||
| _cfStringEncodingConvert(string: self, using: encoding.rawValue, allowLossyConversion: allowLossyConversion) ?? | ||
| // Or attempt an up-call into ICU via FoundationInternationalization | ||
| _icuStringEncodingConvert(string: self, using: encoding, allowLossyConversion: allowLossyConversion) |
There was a problem hiding this comment.
When do we fall back to the ICU one here? On Darwin this encoding is handled via CF, which itself calls into ICU.
There was a problem hiding this comment.
I guess that'll happen if import Foundation has not been done (and CF is present), but import FoundationInternationalization has. I wonder how we can further unify these paths; on Darwin too...
There was a problem hiding this comment.
I had a similar question - I believe _cfStringEncodingConvert will return nil if any of the following are the case:
- CF isn't loaded
- CF is loaded but it doesn't support the provided encoding
- CF is loaded and supports the encoding but the data is malformed
We want the up-call in 2, and I think Tony's question was about 1, but do we want the up call in case 3?
There was a problem hiding this comment.
I now agree that unexpected fallback should be avoided.
Although ICU can support other encodings, I should've made this change focus on EUC-JP.
| } | ||
|
|
||
| extension ICU { | ||
| final class StringConverter: @unchecked Sendable { |
There was a problem hiding this comment.
nit: does this need the @unchecked? I would have assumed that since it's a final class with all immutable, Sendable properties that the compiler can validate this conformance
There was a problem hiding this comment.
Thank you.
That's a vestige of my first implementation where _converter was a bare pointer...
There was a problem hiding this comment.
Sorry.
@unchecked is necessary still because LockedState<OpaquePointer> is not Sendable...
There was a problem hiding this comment.
Ah LockedState is Sendable but it requires its stored value to be of a Sendable type, and I had mistakenly thought OpaquePointer was Sendable but it is indeed not Sendable which is why the Sendable conformance isn't available there. This @unchecked Sendable makes sense then, since despite the pointer and anything it stores never escapes the lock.
Sources/FoundationInternationalization/ICU/ICU+StringConverter.swift
Outdated
Show resolved
Hide resolved
| // Attempt an up-call into swift-corelibs-foundation, which can defer to the CoreFoundation implementation | ||
| _cfStringEncodingConvert(string: self, using: encoding.rawValue, allowLossyConversion: allowLossyConversion) ?? | ||
| // Or attempt an up-call into ICU via FoundationInternationalization | ||
| _icuStringEncodingConvert(string: self, using: encoding, allowLossyConversion: allowLossyConversion) |
There was a problem hiding this comment.
I had a similar question - I believe _cfStringEncodingConvert will return nil if any of the following are the case:
- CF isn't loaded
- CF is loaded but it doesn't support the provided encoding
- CF is loaded and supports the encoding but the data is malformed
We want the up-call in 2, and I think Tony's question was about 1, but do we want the up call in case 3?
Background: EUC-JP is not supported by OSS CoreFoundation, while it is supported by macOS Foundation Framework. See swiftlang#1016 This commit resolves the issue by calling ICU API if necessary.
YOCKOW
force-pushed
the
i-see-you-legacy-string-encodings
branch
from
7ae3f7f to
b0a8981
Compare
|
@swift-ci Please test |
|
@parkera @jmschonfeld Thank you for reviewing. In response to your feedback, I've made some changes:
|
Background: EUC-JP is not supported by OSS CoreFoundation, while it is supported by macOS Foundation Framework.
See #1016
This PR resolves the issue by calling ICU API if necessary.