Cannot compile because of #include <codecvt>?

[vladspbru]

gcc --version
gcc (SUSE Linux) 4.8.5

[zauguin]

@vladspbru I recommend updating gcc.
libstdc++, gcc's C++ standard library, wasn't C++11 feature-complete at version 4.8.5.

@aminroosta But we may want to migrate away from <codecvt> anyway, since it will be deprecated with C++17. Any ideas for alternatives regarding UTF-16 to UTF-8 conversion?

[mjsurette]

Thanks for this awesome library. I can see this helping me greatly in my current project, a genealogy program. I have split off parts of this project into a small library for general use.. https://github.com/mjsurette/easyUtils.

Here you can see how I solved the codecvt issue. Feel free to use it in whole or in part.

[aminroosta]

@mjsurette Thanks for sharing easyUtils, I hope we can find something in the standard library otherwise we can use your implementation as a basis.

@zauguin According to this stackoverflow question std::codecvt template from <locale> itself isn't deprecated. For UTF-8 to UTF-16, there is still std::codecvt<char16_t, char, std::mbstate_t> specialization.

We should try that.

[mjsurette]

Take what I say as nothing more than one more opinion: your code, your rules.

I think that you should use a library, or code your own, if you want code which will be guaranteed portable now and in the future. For now codecvt works in modern compilers. Personally, I have no issues with your library regardless of how you do the conversion. That is no more than an implementation detail. As long as it compiles wherever I need it and works well, I like it.

These are the reasons that I made the decision to write my own solution.

Unicode transformation doesn't belong in <locale> as nothing in converting UTF from one representation to another is/should be locale dependent. It makes me wonder "What are they thinking"? The only reason that I can think of to put it there is to put it near other text handling functions which are locale dependent. ie collate True, this is not just transforming 16 bits to 8 bits. There is some bit shuffling involved. That bit shuffling is the same the world over though, and putting this in locale is somewhat scary.

My reading of http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0618r0.html doesn't make it clear that any codecvt code isn't deprecated. This https://twitter.com/StephanTLavavej/status/837822661094846465 does nothing to reassure me.

The issues cited for deprecation are security, lack of clarity in the spec and non-use. My code throws an exception on bad input, which is what security experts recommend. I have no comment on the other reasons.

Unicode conversion is a straightforward problem involving a transformation of one clearly defined type to another clearly defined type. Templating it does not really make sense. There are several combinations of to/from, but each one is different as it involves a different encoding/decoding. You would be better served with a limited set of encoders and decoders and calling these as necessary. I have implemented the ones necessary for my own use. The only reason I did this was because I couldn't find a suitable library. Honestly, I would have used codecvt had it not been deprecated, despite its other blemishes. In general I trust the STL more than my own code. After all it has been reviewed by people much more experienced and knowledgeable than I am.

Again, just my opinon.

Mike

[zauguin]

@aminroosta I replaced the deprecated stuff with the non-deprecated std::codecvt<> in #143. Apart from migrating away from the deprecated library, it might also fix the original issue
about compatibility with ancient gcc versions.

@mjsurette P0618r0 deprecates all of the <codecvt> header, but this does not include std::codecvt<char16_t, char, std::mbstate_t> because this is part of <locale>, not of <codecvt>.
Actually std::codecvt<...> can't be deprecated because it is reponsible for writing Unicode strings to files with the <iostreams> system.

Personally I would love to see better conversion utilitys in the standard library, but until then I think we should rely on codecvt.

[aminroosta]

@zauguin Awesome ... Thank you so much 👍