GRDB 4 and SQLCipher 4

[groue]

The work on next major version GRDB 4 has started. It is an opportunity to upgrade SQLCipher.

See Upgrading to SQLCipher 4 for more information about the breaking changes introduced by SQLCipher 4.

The SQLCipher 4 repository no longer contains any Xcode project (sqlcipher/sqlcipher#292). This has a big impact on the techniques documented by GRDB for embedding SQLCipher.

• CocoaPods: OK. Upgrading the GRDBCipher podspec so that it depends on SQLCipher 4 should be straightforward.

• Manual installation: NOT OK. The GRDBCipher.xcodeproj project contained in the GRDB repository can no longer embed the non-existing SQLCipher Xcode project.

• Carthage: NOT OK. Even if GRDB does not "officially" support Carthage, some users fork GRDB and set up their customized Carthage support. Those setups will break because the GRDBCipher.xcodeproj project contained in the GRDB repository can no longer embed the non-existing SQLCipher Xcode project.

A few possible options:

1. Go full CocoaPods

   The GRDBCipher.xcodeproj project contained in the GRDB repository would only exist for tests, not for user consumption. It would embed SQLCipher through CocoaPods.

   The only documented and supported way to embed GRDBCipher in an application would be through CocoaPods. Manual installation and Carthage would no longer be supported.

   User damage: atomic, because some options become simply unavailable.

   Setup cost: medium, because we'll be forced to upgrade GRDB 3.6.2 for Swift 5, in order to provide an Xcode upgrade path which does not break GRDB. I was hoping we could avoid this step which is painful because GRDB 3 must support SWIFT_VERSION=4 (details).

   Long run cost: zero.

2. Build a clone of the excellent http://github.com/swiftlyfalling/SQLiteLib for SQLCipher. This would allow GRDB to keep on supporting manual installation, just like custom SQLite builds. But someone has to build and maintain a whole new repository.

   User damage: high because things will break anyway.

   Setup Cost: unknown.

   Long run cost: not zero.

3. Do not upgrade SQLCipher.

   User damage: I don't quite know how to evaluate how much SQLCipher 4 is an improvement over SQLCipher 3.

   Setup Cost: zero.

   Long run cost: zero.

cc @swiftlyfalling and @darrenclark because you may help figuring out the best option.

cc @developernotes and @sjlombardo from SQLCipher, for information.

[darrenclark]

Sorta related to #2, another option might be to move the targets from the old sqlcipher.xcodeproj into GRDBCipher.xcodeproj. Would still have the sqlcipher source code pulled in via the submodule, just the Xcode targets to actually build it would live in groue/GRDB.swift now.

Still is some maintenance cost, but it is primarily just copy/pasting settings from the sqlchipher podspec

I opened a proof of concept PR here: #481

[groue]

Sorta related to #2, another option might be to move the targets from the old sqlcipher.xcodeproj into GRDBCipher.xcodeproj. Would still have the sqlcipher source code pulled in via the submodule, just the Xcode targets to actually build it would live in groue/GRDB.swift now.

That's true!

This solution would not be as flexible as @swiftlyfalling's SQLiteLib. SQLiteLib allows the client app to customize building options without any modification of GRDB and its dependencies, and this is where it really shines.

Yet the current support for SQLCipher does not allow such customization: your suggestion is just as good. I can support it! 👍

Still is some maintenance cost, but it is primarily just copy/pasting settings from the sqlchipher podspec

I'm sure we could handle that in the long run. Once the initial setup is done, I only foresee the updates of SQLCipher sources, and, as you say, keeping an eye on the usual build options.

I opened a proof of concept PR here: #481

That's exactly the kind of bold move I was hoping, you know! Thank you very much, @darrenclark!

I really wish this proof of concept can become production-grade. Let's talk over there!

[charlesmchen-signal]

User damage: I don't quite know how to evaluate how much SQLCipher 4 is an improvement over SQLCipher 3.

One advantage of SQLCipher 4 over 3 is that it is based on a more recent version of SQLite.

Another advantage is that it includes a fix for this issue:

sqlcipher/sqlcipher#255

The issue affects iOS apps using SQLCipher 3 where the database file resides in the iOS "shared container". If an app has an app extension (e.g. a share extension), placing the database file in the "shared container" is the only way for the main app and app extension to share access to a single database. But iOS kills apps that maintain locks on files in the "shared container" while in the background. So iOS would kill your main app every time it enters the background.

[groue]

Thanks @charlesmchen-signal, for reminding this important issue.

@darrenclark, Xcode 10.2 is out now: it would be great to be able to have a working SQLCipher integration soon. Do you think #481 is close to a finish? @Marus would then be able to work on helping users migrate from SQLCipher 3 to SQLCipher 4. And meanwhile I could finish #479 (the main GRDB 4 PR) so that we could ship all this goodness in a nice release :-)

[groue]

Hi folks, here is a report of the current state of affairs regarding SQLCipher 4 and GRDB 4.

1. Allow Database Connection Configuration #508 by @michaelkirk-signal is merged: it makes it possible to open existing SQLCipher 3 databases with GRDB 4 and SQLCipher 4.
2. Move sqlcipher.xcodeproj targets into GRDBCipher.xcodeproj #481 by @darrenclark has been discarded, which means that nobody is currently working on an integration of SQLCipher 4 which looks like the one we had with SQLCipher 3.
3. Because of the previous point, I plan to ship GRDB 4 with CocoaPods as the unique way to use SQLCipher.
4. We still need a SQLCipher 4 transition guide that would explain how to open an existing SQLCipher 3 database with SQLCipher 4, and how to dump the content of an existing SQLCipher 3 database into a new SQLCipher 4 database (guidance).

The points 2 and 3 may create pain for current users of SQLCipher who don't use CocoaPods. It will be possible to address this later, though, if anybody happens to continue the work started in #481. Applications are open!

Contributions for the fourth point, documentation, will also be warmly welcome.

The faster will be the better. Xcode 10.2 is out, and GRDB 4 is late.

[charlesmchen-signal]

@groue Sounds good. Thanks for the update.

[groue]

Superseded by #517