Next #131

onur · 2017-06-07T09:07:55Z

This is first major upgrade to docs.rs

Container OS upgrade

docs.rs is now using Debian "stretch" to build crates. This will allow us to use more recent version of system dependencies.

External crate links

Docs.rs started using a slightly modified rust compiler. This compiler is setting html_root_url for all crates and it's dependencies. This is fixing every single external crate link (along with types, traits etc.) in documentations. To achieve this docs.rs is also using a slightly modified cargo.

You can see this feature in documentation of CargoError implementors. Docs.rs is now resolving every external link properly.

Note: i686-pc-windows-msvc and x86_64-pc-windows-msvc targets are no longer supported, docs.rs will try to build documentation for i686-pc-windows-gnu and x86_64-pc-windows-gnu targets for windows operating system.

Non-default features

This is one of the most requested feature for docs.rs. You can now tell docs.rs to build documentation for the features you want. A simple package.metadata.docs.rs section in your Cargo.toml will do the job:

[package.metadata.docs.rs]
features = [ "feature1", "feature2" ]
all-features = false
no-default-features = true
default-target = "x86_64-unknown-linux-gnu"
rustdoc-args = [ "--example-rustdoc-arg" ]

You can check docsrs-test crate for example.

Note: default-target is not implemented yet.

Shared target directory

Docs.rs no longer builds every single dependency from scratch. It is using a shared deps directory to store all build artifacts. This change improved builds times significantly. A simple comparison from build logs of cargo crate:

cargo old: 111.97 secs
cargo new: 6.48 secs

This allowed me to build entire crates.io-index in 5 days (only the latest versions of crates).

Crate details in navigation

You can see crate details in navigation:

I believe this is providing more essential information in documentations. It looks a bit odd in chrome, hope someone help to make it look better in major browsers.

Transition

Rebuild every single crate (only the latest versions).
Replacement of old cratesfyi-prefix directory with the new one.

Issues

Closes: Allow setting RUSTDOCFLAGS somehow #48
Closes: Set a specific feature during doc generation #50
Closes: Idea: quick dropdown menu to switch between versions #58
Closes: The requested resource does not exist #103
Closes: docs.rs does not produce documentation for sub-crates #128
Partially: Set default target for certain crates, also features #29

This patch is adding `Metadata` type used in `cargo::ops::CompileOptions` to customize docs.rs builds. An example metadata in Cargo.toml: ```text [package] name = "test" [package.metadata.docs.rs] features = [ "feature1", "feature2" ] all-features = true no-default-features = true default-target = "x86_64-unknown-linux-gnu" rustc-args = [ "--example-rustc-arg" ] rustdoc-args = [ "--example-rustdoc-arg" ] dependencies = [ "example-system-dependency" ] ``` This patch is still work in progress and aiming to fix: #29, #48 and #50 - [ ] Save default target to database. - [ ] Install system dependencies before building a package.

This will significantly improve build times but it will use tons of more disk space

Docs.rs is never running rustc on main crate and because of this reason, cargo is ignoring rustdoc-args when rustc-args set. Users must define rustdoc-args to manipulate rustc.

latest_version function is now taking &Vec<String> instead of Json because of versions used in CrateDetails.

retep998 · 2017-06-07T12:33:51Z

Why are i686-pc-windows-msvc and x86_64-pc-windows-msvc no longer supported?

onur · 2017-06-07T12:38:03Z

@retep998 I have to build rust compiler for all tier one platforms myself and it is taking tremendous amount time. Is there any crate that only supports windows-msvc and not windows-gnu? I assumed windows-gnu should be enough for windows related crates.

retep998 · 2017-06-07T13:39:46Z

@onur The one case I can think of where the API of a crate would change is bindings to the CRT. MinGW and VC++ both provide a different CRT, MinGW using the ancient msvcrt.dll with some extra stuff statically linked in, while VC++ has their newer Universal CRT. The set of APIs available differs between those two CRTs, as well as the layouts of various types. That said, neither winapi nor libc actually care about that distinction at the moment, so it's not a huge deal.

Does building the Rust compiler really take that long? I'd imagine you'd simply build the compiler once and have all the tier one platforms configured as targets, and each target only needs a standard library which should only take a few minutes.

steveklabnik · 2017-06-07T15:44:27Z

Is there any crate that only supports windows-msvc and not windows-gnu?

It's ultimately that msvc is the preferred target, so having the docs be for gnu is not great.

gabdube · 2017-06-07T16:23:45Z

Is there any crate that only supports windows-msvc and not windows-gnu? I assumed windows-gnu should be enough for windows related crates.

One of my creates do not build on gnu ( https://github.com/gabdube/native-windows-gui ) . Its only because of a missing symbol (and I can create a workaround) but I guess the deeper you go in the winapi, the more likely these issues can popup.

steveklabnik · 2017-06-07T16:47:20Z

@onur oh one more thing; we should talk about what it would take to get you off of a custom rustdoc. Is there a particular reason you went with it this way instead of sending changes upstream?

onur · 2017-06-07T17:29:12Z

@steveklabnik my rustdoc is basically ignoring html_root_url from crate root and setting it to docs.rs even if its not defined. Also, I might be wrong but rustc only cares about lib.name (--crate-name and --extern NAME=PATH argument passed by cargo). There is no way to get package.name and docs.rs is using a url scheme like: docs.rs/$PACKAGE_NAME/$VERSION/$LIB_NAME. Because of this reason I had to add a new argument to rustc, rustdoc and cargo: --extern-versions. I don't think upstream would accept this changes.

@retep998 unfortunately building windows targets are only possible on a windows host, and apple targets requires apple host etc. I am using 3 different host machine to build rust compiler for all tier 1 platforms. It's just not automated like nightly builds and manual steps are taking too much time.

But if I am not mistaken a windows-gnu host can build windows-msvc targets and it should be easy to add windows-msvc target. I'll add windows-msvc target again when I am free.

steveklabnik · 2017-06-07T19:50:30Z

I don't think upstream would accept this changes.

We might; I'd want to talk about the details, but I'd much prefer that we extend rustdoc for your use-case than have you on a custom rust.

lilianmoraru · 2017-06-10T21:02:35Z

This is pretty awesome work!

You can see crate details in navigation:

Huh, good, I am not blind.
I was always going to Source -> Crate, in order to get to the GitHub page.
I observed the dropdown menu on the navigation a few days ago and was wondering how I didn't observe before this nice menu with all the important info nicely packed in there...

On another note: I see that the metadata supports only a default target.
Would you please consider adding support for multiple targets?
I am thinking of crates like simd here: https://github.com/rust-lang-nursery/simd/blob/3ce0b70860f0ccb6b93002ed66654c76cbfbb234/src/lib.rs#L218 - that currently presents the docs only for x86.
When I looked at the docs, I thought it doesn't support Neon, AVX, etc... I switched between platforms(I was actually searching for AVX, so I was hoping it will pop-up) but it was still presenting only x86(only sse2).

This is related to https://github.com/onur/docs.rs/pull/131

onur · 2017-07-16T21:32:06Z

I re-enabled MSVC targets.

bluss · 2017-07-17T08:17:37Z

Thanks for this. The intercrate linking is also really nice.

onur added 21 commits February 5, 2017 12:20

Add default target to database

8253c50

Ignore rustc_args

ef9e5f3

Bump version

0e1bc93

Use shared target directory when building

163b6d9

This will significantly improve build times but it will use tons of more disk space

Fix build_doc test

d9d84a2

Merge branch 'metadata' into next-metadata

21516cd

rustc-args removed from Metadata

0010939

Docs.rs is never running rustc on main crate and because of this reason, cargo is ignoring rustdoc-args when rustc-args set. Users must define rustdoc-args to manipulate rustc.

Add crate details to rustdoc navigation

10ad1f1

Fix indent and add more links

07484bf

Fix test_latest_versions

bfe1896

latest_version function is now taking &Vec<String> instead of Json because of versions used in CrateDetails.

Use our own version of cargo and rustc

014786e

Merge branch 'use-own-cargo' into next

6cb835e

Merge branch 'shared-target-dir' into next

bedcc0c

Add container name argument

87e4c85

Remove annoying debug messages

251c6cb

Merge branch 'metadata' into next

0dc9b74

Merge branch 'crate-details-in-rustdoc-navigation' into next

c421b33

Fix platform links

bd7464e

Remove crate prefix from dependencies

dfcf1ed

Add target name to platform link

1a95c6e

onur mentioned this pull request Jun 7, 2017

Broken link pointing to seemingly implementation details instead of public interface of a target crate #123

Closed

Release of v0.4.0

71bda02

onur merged commit 19f5839 into master Jun 7, 2017

onur deleted the next branch June 7, 2017 12:00

This was referenced Jun 7, 2017

docs.rs does not produce documentation for sub-crates #128

Closed

Exceptions for setting html_root_url and documentation to docs.rs rust-lang/api-guidelines#78

Closed

gwenn mentioned this pull request Jun 7, 2017

Update doc links to docs.rs rusqlite/rusqlite#182

Merged

lawliet89 referenced this pull request in lawliet89/chrono Jun 26, 2017

Add metadata to include all features for docs on docs.rs

d5aeae0

This is related to https://github.com/onur/docs.rs/pull/131

lawliet89 mentioned this pull request Jun 26, 2017

Add metadata to include all features for docs on docs.rs chronotope/chrono#166

Merged

killercup mentioned this pull request Jun 27, 2017

Add docs.rs metadata to build docs with all features diesel-rs/diesel#973

Closed

Boddlnagg mentioned this pull request Jul 10, 2017

Re-enable MSVC target #135

Closed

danburkert mentioned this pull request Oct 14, 2017

prost-build 0.2.0 docs.rs generation failed tokio-rs/prost#28

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Next #131

Next #131

onur commented Jun 7, 2017 •

edited

Loading

retep998 commented Jun 7, 2017

onur commented Jun 7, 2017

retep998 commented Jun 7, 2017

steveklabnik commented Jun 7, 2017

gabdube commented Jun 7, 2017

steveklabnik commented Jun 7, 2017

onur commented Jun 7, 2017

steveklabnik commented Jun 7, 2017

lilianmoraru commented Jun 10, 2017

onur commented Jul 16, 2017

bluss commented Jul 17, 2017

Next #131

Next #131

Conversation

onur commented Jun 7, 2017 • edited Loading

Container OS upgrade

External crate links

Non-default features

Shared target directory

Crate details in navigation

Transition

Issues

retep998 commented Jun 7, 2017

onur commented Jun 7, 2017

retep998 commented Jun 7, 2017

steveklabnik commented Jun 7, 2017

gabdube commented Jun 7, 2017

steveklabnik commented Jun 7, 2017

onur commented Jun 7, 2017

steveklabnik commented Jun 7, 2017

lilianmoraru commented Jun 10, 2017

onur commented Jul 16, 2017

bluss commented Jul 17, 2017

onur commented Jun 7, 2017 •

edited

Loading