Assert compile error for unspecified array/slice format

[marler8997]

I'm opening an issue based on my comment here: #6870 (comment)

That PR was just merged which means that now the default format for printing slices/arrays will be integer values rather than raw values, i.e.:

const std = @import("std");
pub fn main() void {
    std.debug.warn("Hello, {}\n", .{"World"});
}

$ zig run hello.zig
Hello, [5:0]u8@7ff7cb1be3a0

The fix for this example is to specify {s} whenever you're printing a string. The problem with this is now whenever a format string does not specify the correct format, the bug will not be caught until runtime and the string is inspected. What makes this worse is that this kind of format string is commonly used for error messages, which often don't get tested or inspected until the error occurs, which many times is only seen rarely if at all. There is now a disconnect between what code looks like, and the default way to format strings.

To remedy this, I propose that we require a format specifier for slices/arrays. On top of that, in order to support generic code I propose that we enable today's current behavior of {} with an explicit specifier. Something like {any}. The {any} specifier would mean "print this thing, I'm not picky about the exact format, just print it somehow". The difference here is that instead having this "I'm not picky about the format" attitude by default with {}, the user has to opt-in to it.

Also note that with the PR that was just merged, alot of Zig code will now be broken due to missing format specifiers that were meant to be {s}. Implementing this proposal means these bugs will be found at compile-time so they will be easy to find and fix.

Here's the list of array specifiers:

{s} - format as a list of Unicode characters characters and interpret array as a utf8 encoded string
{d} - format as a list of decimal values
{x} - format as a list of lowercase hex values
{X} - format as a list of uppercase hex values

[daurnimator]

As an alternative solution, what if there was a String type in zig that made it distinct from an array of u8?

[marler8997]

Yeah having distinct string type would be another way to solve this.

[rohlem]

Unless I'm missing something, even if we had a distinct String type, we still want []u8 (any kind of "normal" C string that people forgot to cast) to trigger an assert with the default {} format, right?
Because that seems the essence of this footgun, the fact that the "simplest"/"most obvious" solution gives undesirable output in rarely-tested code paths. Adding a "right way" via an optional String type alone doesn't quite solve this I think.

[marler8997]

@rohlem yes that's a good point. Even if we added a distinct String type, code that uses arrays of u8 would still exist. My guess is that this would be common code meant to work with both strings and data. All this code would still be subject to this problem even if a String type were to exist.

[kyle-github]

If you want to allow easy UX for printing formatted strings, I really only see four possibilities:

1. Do like C and have a char type (it would have to be a Unicode code point I think, so definitely not 8 bits). Arrays of these would be easily inferred to be strings. This is painful for interop with C.
2. Have a utf8 data type for a utf8 byte. It is 8 bits, but when you pass an array of them, you can easily tell and use it as a utf8 string. If it is malformed, you get nasal demons. This works with C pretty well. const hello = "Hello, World."; The type of hello would be [13]utf8.
3. Have a string type and drop down to a []u8 representation when passing to C. This is really a separate string library.
4. Specify that the data should be processed as a string with {s} or something. This is feasible.

Personally I like the second option. If you know that your data is actually a utf8 string, it would be nice to let the system know that via types. It works with sentinel-terminated arrays as well as counted arrays just fine. You should be able to bit cast a byte between utf8 and u8, but they are not the same and should not be trivially interchangeable.

[marler8997]

If you want to allow easy UX for printing formatted strings, I really only see four possibilities:

I wouldn't describe the goal here as "to allow easy UX for printing". The only goal here to catch bugs at compile time instead of runtime. One of the tenants I've learned over the years is:

"interfaces should be easy to use correctly and hard to use incorrectly...and they matter"

- Scott Meyers

I recommend watching this entire talk from Scott Meyers, but here's the timecode of the section where I pulled that quote from:
https://www.youtube.com/watch?v=3WBaY61c9sE&t=43m24s

We've made a change to our fmt.zig module where {} is now the "incorrect" way to format arguments in a common use case, which is when you intend to format raw strings. Note that this change in default behavior isn't the problem. The issue is that now the "easy way" to format arguments with {} produces bugs that are not detected until you run the program and inspect the resulting string. I'm proposing that we modify our interface so this bug is detected at compile-time instead of runtime. The way to make this detectable at compile-time, is to not allow an unspecified formats for these "ambiguous" types. When we detect an "ambiguous" type being formatted with no specifier, its likely the developer has a bug and we can inform them at compile-time. To demonstrate, by making this change to the interface, I've already identified handful of bugs in the compiler (see #7676).

[andrewrk]

Two things this proposal needs to clarify before we can implement it:

• Example of intentionally printing arrays
• Example of intentionally printing any type including arrays

[kyle-github]

HI, @marler8997 I am familiar with Meyers 😸 Perhaps too familiar! I still have his first edition Effective C++ somewhere around here...

Note that that options 2 and 4 are not contradictory. However, what 2 allows is any compile time introspection to determine that something is roughly a printable string without having an explicit string type. Instead utf8 becomes the explicit string element type.

Option 2 would be a significant change to Zig, but it works exactly along Meyer's point. Think of it this way: If you are required to tell your code that a piece of data means something then you want the "interface" for that statement to be easy to use and hard to get wrong.

A printf-style format specifier is out-of-band with the data itself. Because it is out of band, you can change your type but forget to change your specifier. Any time you have to tell code something about your data (metadata) that is not an inherent part of that data itself, you have a risk that the data and metadata diverge. If your metadata is all in your type information, then you cannot lose it unless you transform your type.

I hope that makes it more clear what I was thinking with option 2. It was precisely to make it more difficult to make a mistake that I was thinking of leveraging Zig's excellent type handling and compile time features in order to make strings a little easier to use. This is almost certainly the wrong place to work that out, however, as it is a rather invasive change at this point and this issue is about printing arrays.

[marler8997]

@kyle-github yes I can see the benefits of your proposed option 2. However, adding a new utf8 element type would come with many other effects that I'd have to ponder on. The fact that Zig doesn't distinguish between char and uint8_t like C does makes things simpler. I'm sure Zig could do a better job than C but I myself don't know whether it would be a net gain. So far I haven't seen a big need for it other than for formatting, and I don't think that alone is reason enough to add this new type to the language. I'll keep the potential feature in mind as I continue to work with this language.

P.S. The problems that arise from "out-of-band" data are worth looking into. String interpolation is another solution that gets around it by making the data "in-band". There is a library that allows a form of that, but its a bit verbose.