Figure out in-tree crypto story

[steveklabnik]

Issue by brson
Wednesday Jun 04, 2014 at 22:05 GMT

For earlier discussion, see rust-lang/rust#14655

This issue was labelled with: A-libs in the Rust repository

---

We've previously made the decision not to distribute any crypto with Rust at all, but this is probably not tenable since crypto is used everywhere. My current opinion is that we should not distribute any crypto written in Rust, but that distributing bindings to well-regarded crypto is fine.

Figure out a strategy here, build consensus, then start implementing a robust crypto library out of tree, with the goal of merging into the main distribution someday. There are some existing efforts along these lines that should be evaluated for this purpose.

[dgryski]

PolarSSL will be relicensed from GPL to Apache, making it an alternative to OpenSSL.
http://community.arm.com/groups/internet-of-things/blog/2015/02/09/polarssl-is-dead-long-live-mbed-tls

[burdges]

It's worth mentioning that LLVM cannot create branchless constant time code, so either one should link libraries with assembly imeplementations, or else pursue another compiler option, like https://github.com/klutzy/nadeko perhaps.

See https://moderncrypto.org/mail-archive/curves/2015/000463.html and https://moderncrypto.org/mail-archive/curves/2015/000466.html

[burdges]

There is a simpler question worth addressing too : Is cryptographic key material copyable? Apperently no : rust-lang/rust#20631

We should zero out cryptographic keys when they're dropped, ensuring that LLVM does not optimize it away, as done in sodiumoxide via libsodium : https://github.com/dnaq/sodiumoxide/blob/master/src/newtype_macros.rs

It follows that types containig crpyographic keys must have a drop method that zeros them, and therefore cannot be copyable. It follows that keys will use move semantics, so.. Is it correct that a move will drop what is being moved from?

In practice, we'll commonly use pointers anyways since key material is at least 16 bytes long, but this seemed worth clarifying.

As an aside, we should call mlock and munlock on them too, although that might impact more than I realize. Appears https://github.com/seb-m/tars does eactly this, but implementing full allocators sounds heavy handed. All that's required is a wrapper that calls mlock and munlock like what libsodium does : https://github.com/jedisct1/libsodium-doc/blob/master/helpers/memory_management.md

[burdges]

We should maybe build just a framework for cryptographic crates rather than trying to pack all the cryptographic algorithms into one big crate, so just the basic traits, etc. for handling key material, digests, stream cyphers, block cyphers, boxes, etc. but not the actual algorithms.

[Philipp91]

@burdges I'm currently exploring that idea, starting with hash functions because they're the simplest. I have implementations for these traits in the same crate, which act as a kind of adapter to the actual crypto libraries, which can be swapped similar to rust-native-tls. So it's not the crypto library that needs a dependency on the "framework" crate, as you call it, but the framework crate has a dependency on a single crypto library--which one depends on your cargo features. The end result is similar in that there is a common API, anyone who codes against it can use any implementation, and low-level implementations don't need to live in the same crate.

[burdges]

Isn't that slightly more complicated than making the algorithms depend on the framework crate?

Anyways, there are many crypto algorithm that would benefit from a common base that provides constant time operations. I donno if those should simply be added to core ala #1814 or should be the start of a crypto crate, maybe starting with https://github.com/valarauca/consistenttime or something similar, like util* in https://github.com/DaGenix/rust-crypto/tree/master/src or https://github.com/isislovecruft/curve25519-dalek/blob/master/src/util.rs

[valarauca]

I've made a write up of nadeko's issues. It isn't constant time, and really does not solve the core issues with being constant time. There is a very high level of AST manipulation and understanding of intent that goes into making something constant time. I truely believe we cannot just have it opt in.

I like the idea of a crypto frame work.

Exposing a handful of functions:

• Zero buffer on drop (to ensure this happens 100% of the time)
• Constant time operations (which will just be functions wrapping hand-coded ASM)

This will make crypt-library authors jobs a lot easier.

[Philipp91]

Let's keep two things separate. The first is "under" the implementations of crypto primitives and the second is "on top" of them:

• Helper functions for constant time operations and memory management, which @burdges called "a common base". This is certainly a helpful thing; like @burdges I'm not sure if it should go into the core library or a separate crate. Also, not sure I'd call this one a "framework".
• A high-level API for users ("just the basic traits") that makes it possible to ship an API that users can code against, guaranteeing them that the code using this library will continue to work, without including any actual crypto primitive implementations in the standard library (because there they would be slow to update, etc). This idea was discussed in the previous issue (Figure out in-tree crypto story rust#14655 (comment)). This API, as well, could be in the core library or a rust-lang crate (RFC: policy for rust-lang crates #1242). And it could be a framework (like it is in Java), or it could be less (just a facade, a wrapper, sth. like that).

[burdges]

I said "framework" because:

There are a bunch of useful traits in https://github.com/DaGenix/rust-crypto/tree/master/src for things like hash digests, stream ciphers, encryption and decryption with stream cipher, ctr modes, etc. In principle, one could build upon those, except some design decisions there seem questionably disjoint from std and need either improving or explaining.

Why no in-place operation of stream ciphers? I'd imagine that's just an omission. Should stream ciphers act as iterators though? I donno. If so, should appropriate stream ciphers like ChaCha20 use general purpose iterator machinery for seeking and/or translation from an iterator that operates in blocks. Should digests input stuff that's Hash? etc.

Answering these questions publicly, and/or adjusting traits to do these things, seems important, as otherwise every hash function or cipher developer need to think about these question themselves. It's true constant-time operations are a step below this traits "framework" stuff, but they are similar in that anyone writing crypto primitives needs to think about them if they are not being done right elsewhere.

[Philipp91]

I see your point of saving crypto library implementors from reinventing the wheel all the time. That's a major benefit of such a "framework".
Another benefit is that it's one single API for crypto users, similar to JCA in Java. And, hopefully, it's a pretty good API because a lot of work went into designing it by clarifying questions like the one you raised.

While a library implementor could use both the outward-facing traits and the low-level helpers, the crypto user should better not be able to access the low-level stuff directly, so it should at least be in a separate module. And I argue it should be in a separate crate because not all crypto libraries really need it. There are quite many which are actually just wrappers of some C library (OpenSSL, BoringSSL, libsodium), so they wouldn't need the constant-time operations in Rust. Instead, they are based on other (common) low-level things like libc, some FFI generator or something like that. And they work under the same high-level traits as the crypto APIs implemented in Rust itself.

And I agree that it's good to answer these questions publicly. Maybe, the API/traits and the lower-level functions are two separate discussions, though, because there are different stakeholders. And probably neither of the discussions belongs into this thread, but I would have many details to discuss already :-) including iterators in cipher APIs.

You name rust-crypto as a base that could be adapted. Have you also looked at others? This topic has been discussed in the #rust-crypto IRC a few weeks back and it seemed to me that ring's API is more well-regarded.

[burdges]

A few constant time building blocks sound small compared to say pulling in an extra hash function, much less AES and RSA, like every existing crypto library does.

Aren't those Algorithm structs in ring too confining for a general purpose crypto library? Those fit well both with ring's history in BoringSSL of course, and more generally for providing TLS, but a more rust-like API would probably be Algorithm::new(..) and traits, like rust-crypto.

I'd say leave any "stakeholders" who want uber-safety to using ring and sodiumoxide. Those projects make a range of safety decisions, including evaluating if the Rust code is mature enough to replace solid C code. Rust is not lacking in foot gun free crypto libraries.

Rust is weaker on flexible building blocks though:

We want to simplify building a separate crate to provide a crypto primitive, without needing to dig too deeply into rustc or LLVM. Right now, people send pull requests to rust-crypto, but that project should not be responsible for everyone's pet cipher or hash function. I suppose just breaking rust-crypto in a crypto-current and crypto-legacy crates would require exposing enough building blocks that new ciphers and hash functions need not send pull requests.

We want to simplify using such crates in protocols, including unconventional protocols involving stuff like blind signatures, Axolotl, elligator, semi-private keys, mixnets, ECQV, post-quantum, etc. We should not worry about public-key algorithms right now, as stand alone crates do those well enough, but the symmetric crypto should not stand in the way when people when do unusual asymmetric stuff.

Applications are increasingly needing these unconventional usages. If you insist on avoiding all foot guns, then either important new things get built in languages like Go, or else everyone maintains their own private fork of the crypto library. And I doubt you'll do as good a job of being foot gun free as ring anyways.

In this vein, curve25519-dalek is the right approach to doing a public key library. All the mathematical submodules are well documented and exported. Fixed-size arrays are used everywhere with the help of the arrayref crate, so no slices. etc.

[tarcieri]

@Philipp91

the framework crate has a dependency on a single crypto library--which one depends on your cargo features. The end result is similar in that there is a common API, anyone who codes against it can use any implementation, and low-level implementations don't need to live in the same crate.

This approach makes less sense to me than having a single set of traits multiple crypto libraries can target. It would require that libraries which want to buy into a common crypto core be rewritten to use those traits, but to me, that's the "right answer", as opposed to trying to make a one-size-fits-all wrapper to the divergent APIs present in existing crypto libraries.

As @burdges noted, there are already some good traits which could be extracted for this purpose in rust-crypto.

As a former JCA user, it was, like many Java things, an extremely overcomplicated and bad abstraction (the entire Java crypto ecosystem is the stuff of nightmares). It had somewhat noble goals, but from a practical perspective was a giant pain to work with.

I would prefer an approach that leans on Rust's type system more, with simple abstractions built on common traits, and no need to lean on cargo features: just include the crates with the algorithms you want, all built on a common set of traits, and everything just works. I think this approach could potentially still enable things like "ask for a named instance of a cipher at runtime"

That way, crappy algorithms people shouldn't be using can be spun out into separate crates, complete with appropriate "caveat emptor" warnings for their users.

[tarcieri]

@burdges

As an aside, we should call mlock and munlock on them too, although that might impact more than I realize. Appears [tars] does eactly this

There are now a litany of incompatible Rust libraries for memory protected secret storage:

• tars
• memsec
• secrets
• secstr
• region

If nothing else, it seems like something else that could benefit from a common trait.

but implementing full allocators sounds heavy handed. All that's required is a wrapper that calls mlock and munlock like what libsodium does : https://github.com/jedisct1/libsodium-doc/blob/master/helpers/memory_management.md

libsodium does more than that: it adds guard pages at the beginning and end of the region using mprotect() to protect against memory scraping or Heartbleed-style attacks. Ideally for something like this you'd have a sort of arena you allocate once for n-keys so you can store many in the same buffer.

Another popular approach is to use a master key-encrypting-key (KEK), storing all secrets encrypted in memory, and using the KEK to decrypt them on demand for a particular operation.

All that said, this really looks to me like another case that can benefit from a common trait, and depending on your paranoia level you can pick the exact strategy/crate you want to safeguard your keys/other secrets.