Added embedded bitcode support for ios #1564

evant · 2018-05-06T00:17:20Z

This introduces the flags '--embed_bitcode' and '--embed_bitcode_marker' to
make it possible to build bitcode-enabled frameworks.

Fixes #1202, #1408

olonho · 2018-05-06T08:15:53Z

...native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/llvm/IrToBitcode.kt

+    val bytes = if (marker) {
+        byteArrayOf()
+    } else {
+        val buf = LLVMWriteBitcodeToMemoryBuffer(context.llvmModule)


buf -> buffer (we keep unshortened variable names in K/N).

olonho · 2018-05-06T08:17:35Z

...native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/llvm/IrToBitcode.kt

    if (context.shouldContainDebugInfo()) {
        DIFinalize(context.debugInfo.builder)
    }
 }

+private fun embedBitcode(context: Context, marker: Boolean) {
+    val isIos = context.config.platform.configurables.target.family == Family.IOS


Hmm, why IOS is specific here?

Taken from rust-lang/rust@0e0f74b#diff-a3b24dbe2ea7c1981f9ac79f9745f40aR851 and https://github.com/llvm-mirror/clang/blob/master/lib/CodeGen/BackendUtil.cpp#L1237
the section name is just different on that platform. Unsure if this should be on macos as well?

It's Apple specific - so isApple maybe?

SvyatoslavScherbina · 2018-05-09T01:08:00Z

backend.native/cli.bc/src/org/jetbrains/kotlin/cli/bc/K2NativeCompilerArguments.kt

@@ -16,6 +16,7 @@

 package org.jetbrains.kotlin.cli.bc

+import org.jetbrains.kotlin.backend.konan.EmbedBitcode


This import seems to be redundant.

SvyatoslavScherbina · 2018-05-09T01:08:52Z

backend.native/cli.bc/src/org/jetbrains/kotlin/cli/bc/K2NativeCompilerArguments.kt

@@ -139,5 +140,10 @@ class K2NativeCompilerArguments : CommonCompilerArguments() {
    @Argument(value = "--verify_ir", description = "Verify IR")
    var verifyIr: Boolean = false

+    @Argument(value = "--embed_bitcode", description = "embed LLVM bitcode in object files")


It makes sense to enable bitcode embedding by default on macOS and iOS targets.
@olonho WDYT?

How much bigger binaries would become then?

The smallish sample project I was testing with went from 153MB to 238MB

SvyatoslavScherbina · 2018-05-09T01:09:36Z

...native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/llvm/IrToBitcode.kt

@@ -130,11 +131,49 @@ internal fun emitLLVM(context: Context) {
        irModule.acceptVoid(codegenVisitor)
    }

+    when (context.shouldEmbedBitcode()) {


Is bitcode embedding performed before debug info generation intentionally?

SvyatoslavScherbina · 2018-05-09T01:11:47Z

...native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/llvm/IrToBitcode.kt

+            name = "konan.embedded.module",
+            elemType = int8Type,
+            elements = bytes.map { Int8(it) },
+            isExported = true


What's the purpose of passing isExported = true and then setting linkage to private?
Passing isExported = true just sets linkage to external (i.e. public).

SvyatoslavScherbina · 2018-05-09T01:13:36Z

backend.native/compiler/ir/backend.native/src/org/jetbrains/kotlin/backend/konan/LinkStage.kt

@@ -176,7 +179,13 @@ internal class LinkStage(val context: Context) {
                KonanTarget.MACOS_X64 -> "Versions/A/$dylibName"
                else -> error(target)
            }
-            frameworkLinkerArgs = listOf("-install_name", "@rpath/${framework.name}/$dylibRelativePath")
+            val args = mutableListOf("-install_name", "@rpath/${framework.name}/$dylibRelativePath")
+            when (context.shouldEmbedBitcode()) {


Shouldn't the same args be passed to linker when compiling the program instead of framework?

This introduces the flags '--embed_bitcode' and '--embed_bitcode_marker' to make it possible to build bitcode-enabled frameworks.

olonho · 2018-05-13T08:15:31Z

It seems that bitcode embedding need to be somewhat trickier than that, as we need to ensure, that what's get embedded is actual bitcode after link time optimizations, at least dead code elimination, otherwise code grows awfully large. So we will investigate possibilities here.

SvyatoslavScherbina · 2018-05-18T08:45:31Z

We made some investigations. There are problems (besides mentioned above) with bitcode embedding that aren't solved by this pull request.

To emit bitcode our compiler uses LLVM 5.0.0 built from original sources, but Xcode toolchain uses LLVM with custom patches, different versioning scheme and unclear update policy. So to emit proper bitcode we have to use Xcode's LLVM.
Our resulting binaries contain code built from assembly, e.g. in libffi. It is OK on iOS, but doesn't seem to be permitted on watchOS.

So bitcode embedding on watchOS can't be supported by this PR.
As for iOS, it is possible to make binary look like it has bitcode embedded, without really embedding bitcode that we emit when compiling Kotlin code, and we are considering to implement this trick.

dnedrow · 2018-05-18T11:29:22Z

As for iOS, it is possible to make binary look like it has bitcode embedded, without really embedding bitcode…

What are the implications when a bitcode enabled app is uploaded to the AppStore, but any include K/N code isn’t actually bitcode enabled?

Bitcode is enabled by default for new Xcode projects.

SvyatoslavScherbina · 2018-05-18T11:35:50Z

@dnedrow the implications would be the same as when Xcode project has assembly files. Assembly files are permitted on iOS even when bitcode is enabled. So this solution doesn't seem to cause any observable implications. Xcode is able to recompile an archive with such an application as any other bitcode-enabled application archive.

olonho · 2018-06-26T08:30:25Z

Seems this way valid bitcode cannot be generated, so let's close this one for now, and come back again, when focus on that feature again.

evant force-pushed the bitcode-support branch from a84c7b3 to 6cb486c Compare May 6, 2018 00:38

olonho reviewed May 6, 2018

View reviewed changes

evant force-pushed the bitcode-support branch from 6cb486c to 5e9e88b Compare May 7, 2018 12:45

SvyatoslavScherbina reviewed May 9, 2018

View reviewed changes

Added embedded bitcode support for ios

3625d77

This introduces the flags '--embed_bitcode' and '--embed_bitcode_marker' to make it possible to build bitcode-enabled frameworks.

evant force-pushed the bitcode-support branch from 5e9e88b to 3625d77 Compare May 9, 2018 17:31

ngocdaothanh mentioned this pull request May 23, 2018

Add Bitcode support for iOS flutter/flutter#15288

Closed

olonho closed this Jun 26, 2018

noheltcj mentioned this pull request Aug 11, 2018

Enable bitcode support for ios arm64 architecture noheltcj/multi-native#6

Merged

viteinfinite mentioned this pull request Sep 30, 2018

Update LLVM to 6.0.1 and WebAssembly target refactoring #1788

Merged

oleksandr-yefremov mentioned this pull request Feb 20, 2019

Support Apple app store bitcode rust-lang/rust#35968

Closed

tmm1 mentioned this pull request Mar 13, 2019

x/mobile: add bitcode support golang/go#22395

Closed

		@@ -16,6 +16,7 @@

		package org.jetbrains.kotlin.cli.bc

		import org.jetbrains.kotlin.backend.konan.EmbedBitcode

Added embedded bitcode support for ios #1564

Added embedded bitcode support for ios #1564

Uh oh!

Conversation

evant commented May 6, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

evant May 6, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

olonho commented May 13, 2018

Uh oh!

SvyatoslavScherbina commented May 18, 2018

Uh oh!

dnedrow commented May 18, 2018

Uh oh!

SvyatoslavScherbina commented May 18, 2018

Uh oh!

olonho commented Jun 26, 2018

Uh oh!

Uh oh!

evant commented May 6, 2018 •

edited

Loading

evant May 6, 2018 •

edited

Loading