Windows Console and Unicode

[goderbauer]

It seems like Dart cannot properly print unicode characters to the Windows console (cmd.exe or powershell.exe seem to be equally affected). Example test.dart Program:

main() {
  print("These are three black spades: ♠♠♠");
}

Actual console output of dart test.dart:

These are three black spades: ΓÖáΓÖáΓÖá

Ideally, this should have shown the three spades, or - in case those are not available in the font used in the console - it should have show Unicode's missing character symbol (e.g. �).

While researching this issue on the web I realized that properly printing Unicode to the Windows console is actually fairly hard to do. However, it seems like the people over at Python just figured out how to properly do this in python 3.6.0, which was just released last month. Example test.py program:

#!/usr/bin/python
# -*- coding: utf-8 -*-

print("These are three black spades: ♠♠♠")

Actual console output of python test.py (requires python 3.6.0!):

These are three black spades: ♠♠♠

Yeah! \o/

It would be most awesome-st if Dart could also fix the unicode printing behavior on Windows. That would really help us in Flutter, where we make extensive use of unicode in console output (flutter/flutter#7714).

In case it is helpful, here are the relevant discussions around fixing this in python:

• http://bugs.python.org/issue1602 - bug discussing the issue (has been open for 7 years, relevant bits for fixing the issue are towards the end)
• https://www.python.org/dev/peps/pep-0528/ - Python Enhancement Proposal describing the fix
• https://hg.python.org/cpython/rev/6142d2d3c471 - commit including the fix

From the python discussions, it seems like WriteConsoleW() from the Windows API should be used to write strings to the console in order to get unicode support.

[kevmoo]

@nex3 has dealt with this before -- may have some more context

[nex3]

I just avoid writing Unicode characters on Windows 😞. If we could find a way to make them work, that would be rad.

[mit-mit]

I gave this a try on Windows using C#:

        static void Main(string[] args)
        {
            Console.WriteLine("These are three black spades: ♠♠♠");
        }

This did not work; the output was:

However, if I configure the encoding property on the console:

        static void Main(string[] args)
        {
            Console.OutputEncoding = System.Text.Encoding.UTF8;
            Console.WriteLine("These are three black spades: ♠♠♠");
        }

it works:

[mit-mit]

https://msdn.microsoft.com/en-us/library/system.console.outputencoding

[goderbauer]

Thanks for looking at it. It looks like both of your code samples (working and not working) are the same. To avoid future confusion, it might be worth correcting that :)

[mit-mit]

Sorry, fixed

[goderbauer]

Is it possible to set the OutputEncoding of the Console to UTF8 from dart to fix this problem?

[fsc8000]

I think yes, but it's a bigger change to the libraries because it would have to be done in an platform-independent way in the io library.

For Windows as a stop-gap, you could try running chcp 65001 from your Dart program. (and save the currently active code page first using just chcp to restore it before exit) It's not the desired solution, but may get you running.

When I do this it prints the spades correctly:

import 'dart:io';
main() {
  Process.runSync("chcp", ["65001"], runInShell: true);  // Or the async version Process.run
  print("These are three black spades: ♠♠♠");
  stdout.writeln("These are three black spades: ♠♠♠");
}

Output:

C:\src\dartgit\sdk>out\ReleaseX64\dart uni.dart
These are three black spades: ♠♠♠
These are three black spades: ♠♠♠