Skip to content

manual pre-generation: separate .etex files from stdlib build #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 1, 2016
Merged

manual pre-generation: separate .etex files from stdlib build #1

merged 1 commit into from
Sep 1, 2016

Conversation

@gasche
Copy link

@gasche gasche commented Aug 8, 2016

We introduce a pregen-etex Makefile rule that only build the .etex
files of the manual (whose OCaml examples may be tested and expected
to produce certain outputs), without also building the
standard-library documentation. This is faster than the previous
pregen rule as ocamldoc on the standard library was the
bottleneck. On my machine, pregen-etex completes in 1.5s-2s, while
pregen completes in 6s-7s.

@gasche
manual pre-generation: separate .etex files from stdlib build
81743e9 
We introduce a `pregen-etex` Makefile rule that only build the .etex
files of the manual (whose OCaml examples may be tested and expected
to produce certain outputs), without also building the
standard-library documentation. This is faster than the previous
`pregen` rule as ocamldoc on the standard library was the
bottleneck. On my machine, `pregen-etex` completes in 1.5s-2s, while
`pregen` completes in 6s-7s.
@gasche gasche mentioned this pull request Aug 8, 2016
Merged
@Octachron Octachron merged commit 81743e9 into Octachron:manual_guarded_caml_example Sep 1, 2016
Octachron pushed a commit that referenced this pull request Jun 14, 2017
@jeremiedimino
Merge pull request #1 from dra27/prefix-the-stdlib
abcfb85 
Two native Windows fixes for GPR#1010
Octachron pushed a commit that referenced this pull request Feb 3, 2018
@jfehrle
Merge pull request #1 from Octachron/pr/1545
2bd9949 
manual: last minute  fixes
Drup pushed a commit that referenced this pull request Aug 27, 2020
@shindere
Split Inria CI's extra-checks job, take #1
c89c464 
This job did actually do two different things:

1. Check that the compiler can be built without the world.opt target

2. Run sanitizers

This commit thus splits the extra-checks job into two separate
ones that are defined as Jenkins pipeline jobs named sanitizers and
step-by-step-build.
Drup pushed a commit that referenced this pull request Aug 27, 2020
@shindere
Use a pipeline to define the bootstrap Jenkins job, take #1
d0d8ace
Drup pushed a commit that referenced this pull request Aug 27, 2020
@shindere
Define Inria CI's Risc-V job as a Jenkins pipeline, take #1
5cdc807
Drup pushed a commit that referenced this pull request Aug 27, 2020
@shindere
Define Inria CI's other-configs job as a Jenkins pipeline, take #1
b1ffaf6
Octachron added a commit that referenced this pull request Sep 2, 2021
@Octachron
manual: more explicit include of module documentation in latex mode
5fe6165 
The macro \libdocitem{Bigarray} was including `refman/extension/bigarray.etex`.
Rewriting the macro to input `library/#1.tex` instead of `#1.tex`
avoids this issue and should be hopefully less brittle in the future.
Octachron pushed a commit that referenced this pull request Jul 26, 2024
@gasche @edwintorok
fix(toplevel): avoid hashing uninitialized values when printing (ocam…
d33d999 
…l#13294)

The toplevel printer detects cycles by keeping a hashtable of values
that it has already traversed.

However, some OCaml runtime types (at least bigarrays) may be
partially uninitialized, and hashing them at arbitrary program points
may read uninitialized memory. In particular, the OCaml testsuite
fails when running with a memory-sanitizer enabled, as bigarray
printing results in reads to uninitialized memory:

```
==133712==WARNING: MemorySanitizer: use-of-uninitialized-value
    #0 0x4e6d11 in caml_ba_hash /var/home/edwin/git/ocaml/runtime/bigarray.c:486:45
    #1 0x52474a in caml_hash /var/home/edwin/git/ocaml/runtime/hash.c:251:35
    #2 0x599ebf in caml_interprete /var/home/edwin/git/ocaml/runtime/interp.c:1065:14
    #3 0x5a909a in caml_main /var/home/edwin/git/ocaml/runtime/startup_byt.c:575:9
    #4 0x540ccb in main /var/home/edwin/git/ocaml/runtime/main.c:37:3
    #5 0x7f0910abb087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #6 0x7f0910abb14a in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #7 0x441804 in _start (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x441804) (BuildId: 7a60eef57e1c2baf770bc38d10d6c227e60ead37)

  Uninitialized value was created by a heap allocation
    #0 0x47d306 in malloc (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x47d306) (BuildId: 7a60eef57e1c2baf770bc38d10d6c227e60ead37)
    #1 0x4e7960 in caml_ba_alloc /var/home/edwin/git/ocaml/runtime/bigarray.c:246:12
    #2 0x4e801f in caml_ba_create /var/home/edwin/git/ocaml/runtime/bigarray.c:673:10
    #3 0x59b8fc in caml_interprete /var/home/edwin/git/ocaml/runtime/interp.c:1058:14
    #4 0x5a909a in caml_main /var/home/edwin/git/ocaml/runtime/startup_byt.c:575:9
    #5 0x540ccb in main /var/home/edwin/git/ocaml/runtime/main.c:37:3
    #6 0x7f0910abb087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #7 0x7f0910abb14a in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #8 0x441804 in _start (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x441804) (BuildId: 7a60eef57e1c2baf770bc38d10d6c227e60ead37)

SUMMARY: MemorySanitizer: use-of-uninitialized-value /var/home/edwin/git/ocaml/runtime/bigarray.c:486:45 in caml_ba_hash
```

The only use of hashing in genprintval is to avoid cycles, that is, it
is only useful for OCaml values that contain other OCaml values
(including possibly themselves). Bigarrays cannot introduce cycles,
and they are always printed as "<abstr>" anyway.

The present commit proposes to be more conservative in which values
are hashed by the cycle detector to avoid this issue: we skip hashing
any value with tag above No_scan_tag -- which may not contain any
OCaml values.

Suggested-by: Gabriel Scherer <[email protected]>

Signed-off-by: Edwin Török <[email protected]>
Co-authored-by: Edwin Török <[email protected]>
Octachron pushed a commit that referenced this pull request Aug 26, 2024
@edwintorok
runtime_events_consumer: fix uninitialized data reads
6857e5b 
Found by -fsanitize=memory -fsanitize-memory-track-origins:
```
> ==102752==WARNING: MemorySanitizer: use-of-uninitialized-value
>     #0 0x7f2ba7fb4ea4 in caml_runtime_events_read_poll /var/home/edwin/git/ocaml/otherlibs/runtime_events/runtime_events_consumer.c:496:18
>     #1 0x7f2ba7fbc016 in caml_ml_runtime_events_read_poll /var/home/edwin/git/ocaml/otherlibs/runtime_events/runtime_events_consumer.c:1207:9
>     ocaml#2 0x59ba5c in caml_interprete /var/home/edwin/git/ocaml/runtime/interp.c:1058:14
>     ocaml#3 0x5a9220 in caml_main /var/home/edwin/git/ocaml/runtime/startup_byt.c:575:9
>     ocaml#4 0x540d6b in main /var/home/edwin/git/ocaml/runtime/main.c:37:3
>     ocaml#5 0x7f2ba8120087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
>     ocaml#6 0x7f2ba812014a in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
>     ocaml#7 0x441804 in _start (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x441804) (BuildId: 617637580ee48eff08a2bce790e1667ad09f3b69)
>
>   Uninitialized value was stored to memory at
>     #0 0x7f2ba7fb4e9d in caml_runtime_events_read_poll /var/home/edwin/git/ocaml/otherlibs/runtime_events/runtime_events_consumer.c:497:69
>     #1 0x7f2ba7fbc016 in caml_ml_runtime_events_read_poll /var/home/edwin/git/ocaml/otherlibs/runtime_events/runtime_events_consumer.c:1207:9
>     ocaml#2 0x59ba5c in caml_interprete /var/home/edwin/git/ocaml/runtime/interp.c:1058:14
>     ocaml#3 0x5a9220 in caml_main /var/home/edwin/git/ocaml/runtime/startup_byt.c:575:9
>     ocaml#4 0x540d6b in main /var/home/edwin/git/ocaml/runtime/main.c:37:3
>     ocaml#5 0x7f2ba8120087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
>     ocaml#6 0x7f2ba812014a in __libc_start_main@GLIBC_2.2.5 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
>     ocaml#7 0x441804 in _start (/var/home/edwin/git/ocaml/runtime/ocamlrun+0x441804) (BuildId: 617637580ee48eff08a2bce790e1667ad09f3b69)
>
>   Uninitialized value was created by an allocation of 'buf' in the stack frame
>     #0 0x7f2ba7fb3dbc in caml_runtime_events_read_poll /var/home/edwin/git/ocaml/otherlibs/runtime_events/runtime_events_consumer.c:402:7
>
```

This is in fact an EV_LIFECYCLE with EV_RING_STOP, which has 0
additional data, and thus msg_length 2:
```
runtime/runtime_events.c:      EV_RUNTIME, (ev_message_type){.runtime=EV_LIFECYCLE}, EV_RING_STOP, 0,
```

Attempting to read from `buf[2]` would read uninitialized data.

Signed-off-by: Edwin Török <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@gasche @Octachron