Error Headers Unexpected char

[jerson]

Fatal Exception: java.lang.IllegalArgumentException: Unexpected char 0xae at 47 in user-agent value: Dalvik/1.6.0 (Linux; U; Android 4.4.2; INCO AIR® Build/KOT49H)
       at okhttp3.Headers$Builder.checkNameAndValue(Headers.java:283)
       at okhttp3.Headers$Builder.add(Headers.java:233)
       at com.facebook.react.modules.network.NetworkingModule.extractHeaders(NetworkingModule.java:469)
       at com.facebook.react.modules.network.NetworkingModule.sendRequest(NetworkingModule.java:180)
       at java.lang.reflect.Method.invokeNative(Method.java)
       at java.lang.reflect.Method.invoke(Method.java:515)
       at com.facebook.react.bridge.BaseJavaModule$JavaMethod.invoke(BaseJavaModule.java:319)
       at com.facebook.react.bridge.NativeModuleRegistry$ModuleDefinition.call(NativeModuleRegistry.java:187)
       at com.facebook.react.bridge.NativeModuleRegistry.call(NativeModuleRegistry.java:62)
       at com.facebook.react.bridge.CatalystInstanceImpl$NativeModulesReactCallback.call(CatalystInstanceImpl.java:432)
       at com.facebook.react.bridge.queue.NativeRunnableDeprecated.run(NativeRunnableDeprecated.java)
       at android.os.Handler.handleCallback(Handler.java:808)
       at android.os.Handler.dispatchMessage(Handler.java:103)
       at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java:31)
       at android.os.Looper.loop(Looper.java:193)
       at com.facebook.react.bridge.queue.MessageQueueThreadImpl$3.run(MessageQueueThreadImpl.java:196)
       at java.lang.Thread.run(Thread.java:841)

[swankjesse]

Strictly speaking this is a problem with Facebook’s NetworkingModule.java. The fix is a method like so:

  /** Returns {@code s} with control characters and non-ASCII characters replaced with '?'. */
  public static String toHumanReadableAscii(String s) {
    for (int i = 0, length = s.length(), c; i < length; i += Character.charCount(c)) {
      c = s.codePointAt(i);
      if (c > '\u001f' && c < '\u007f') continue;

      Buffer buffer = new Buffer();
      buffer.writeUtf8(s, 0, i);
      for (int j = i; j < length; j += Character.charCount(c)) {
        c = s.codePointAt(j);
        buffer.writeUtf8CodePoint(c > '\u001f' && c < '\u007f' ? c : '?');
      }
      return buffer.readUtf8();
    }
    return s;
  }

[jerson]

ok , I'm using native react and temporarily solve it with this code

const userAgent = DeviceInfo.getUserAgent().replace(/[^\w\s./;()]/gi, '');

[aheadlcx]

@swankjesse
hi , I meet this when using React Native 0.44 . Why do you say this is a problem with Facebook’s NetworkingModule.java . The crash code is in OkHttp Library . Could you explain this .

[yschimke]

react native is building an invalid custom user agent from the device platform info without sanitising the possible characters.

[aheadlcx]

@yschimke
Hi , thanks for your answer . I provide an similar crash message. I get the field named android.os.Build.MODEL and add it to http header , and receive follow crash message .

java

java.lang.IllegalArgumentException: Unexpected char 0x5927 at 4 in phone_type value: Md6-大麦
	at okhttp3.Headers$Builder.checkNameAndValue(Headers.java)
	at okhttp3.Headers$Builder.add(Headers.java)
	at okhttp3.Request$Builder.addHeader(Request.java)
	at com.packagename.network.reactnative.NetworkModule.addDeviceInfoIntoHeader(NetworkModule.java)
	at com.packagename.network.reactnative.NetworkModule.GET(NetworkModule.java)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.facebook.react.bridge.BaseJavaModule$JavaMethod.invoke(BaseJavaModule.java)
	at com.facebook.react.cxxbridge.JavaModuleWrapper.invoke(JavaModuleWrapper.java)
	at com.facebook.react.bridge.queue.NativeRunnable.run(Native Method)
	at android.os.Handler.handleCallback(Handler.java:815)
	at android.os.Handler.dispatchMessage(Handler.java:104)
	at com.facebook.react.bridge.queue.MessageQueueThreadHandler.dispatchMessage(MessageQueueThreadHandler.java)
	at android.os.Looper.loop(Looper.java:207)
	at com.facebook.react.bridge.queue.MessageQueueThreadImpl$3.run(MessageQueueThreadImpl.java)
	at java.lang.Thread.run(Thread.java:818)

[aheadlcx]

@yschimke
Thanks you very much.
And I have a question. The code which throw Exception is in OkHttp library . The okhttp3.Headers.java .Why do you think this issue belong to react native .

    private void checkNameAndValue(String name, String value) {
      if (name == null) throw new NullPointerException("name == null");
      if (name.isEmpty()) throw new IllegalArgumentException("name is empty");
      for (int i = 0, length = name.length(); i < length; i++) {
        char c = name.charAt(i);
        if (c <= '\u001f' || c >= '\u007f') {
          throw new IllegalArgumentException(Util.format(
              "Unexpected char %#04x at %d in header name: %s", (int) c, i, name));
        }
      }
      if (value == null) throw new NullPointerException("value == null");
      for (int i = 0, length = value.length(); i < length; i++) {
        char c = value.charAt(i);
        if (c <= '\u001f' || c >= '\u007f') {
          throw new IllegalArgumentException(Util.format(
              "Unexpected char %#04x at %d in %s value: %s", (int) c, i, name, value));
        }
      }
    }

Thanks again.

[yschimke]

See the comment above by Jesse with toHumanReadableAscii method.

I'm pretty confident that in internal code Facebook must clean the user agent before setting.

[aheadlcx]

@yschimke
Yes , ths toHumanReadableAscii method by Jesse can avoid this crash . And what this method do is filtered non-ASCII characters .
This crash will arise when invoking okhttp3.Headers.Builder.checkNameAndValue method with non-ASCII value . So this crash has nothing to do with setting user agent .

[yschimke]

The original report is for

Unexpected char 0xae at 47 in user-agent value: Dalvik/1.6.0 (Linux; U; Android 4.4.2; INCO AIR® Build/KOT49H)

Your is for

Unexpected char 0x5927 at 4 in phone_type value: Md6-大麦

So please apply to the same method to your "phone_type" header

[megantracy93]

The solutions given here are just to replace the invalid characters with valid ones. However, the list of allowed characters is extremely small and basically only supports english characters.

When using this library in other locales (like Chinese or Arabic), sanitizing all of the "invalid" characters can result in a poor user experience. Is there any chance that support for these characters could actually be added to this library?

Or are there any other suggested solutions that result in actually supporting these characters, rather than just removing them?

[swankjesse]

@megantracy93 end-users shouldn’t see user agent strings? What do web browsers do?

[megantracy93]

Currently we are using OkHttp (via React Native) to perform file uploads, and we put the file name in the header values. Thus, the file name must be limited to this char set, so any user that uploads a file with non-english characters in the filename would have their filename stripped of its meaning & uniqueness.

On Web & iOS these characters seem to be supported as they are, but on Android they are not.

[swankjesse]

Ooh interesting.

Could you provide an example filename? Do iOS and web support full Unicode characters like “Starfox 零” or just ISO-8859-1 characters?

Can you capture one of these headers from iOS or web? I’m curious as to whether they’re using ISO-8859-1 (which supports many western-languages, but not many languages) or UTF-8 (which is not allowed according to the spec). It’s possible that there’s a custom encoding happening here, though I don’t know of one.