Formats set in _metadata.yml not overridden by file-level YAML header

[petrbouchal]

Bug description

If multiple formats are specified in a _metadata.yml, all are rendered even if one attempts to override the format setup via the file-level YAML header.

The use case for me is as follows:

• in a subdirectory, I have an index file and several longer pages
• I want the pages accessible in multiple format
• but don't need the index page in all these formats, just HTML

(I am aware of the workaround by setting format-links: false), but it seems wasteful to generate these unneeded files and the workaround may be insufficient for more complex combination of formats. I also know I could set all the formats in each file, but that seems counter to the logic of having a metadata file.)

Steps to reproduce

1. new project

quarto create-project --type website

2. update formats in quarto.yml

project:
  type: website

website:
  title: "quarto-formats"
  navbar:
    left:
      - href: index.qmd
        text: Home
      - about.qmd

format:
  html:
    theme: cosmo
    css: styles.css
    toc: true
  docx: default
  pdf: default

index.qmd at root level stays as created by quarto:

---
title: "quarto-formats"
format: 
    html: default
---

This is a Quarto website.

To learn more about Quarto websites visit <https://quarto.org/docs/websites>.

3. add subdirectory and metadata file

_metadata.yml in a subdirectory

format:
  html: default
  docx: default

4. create index.qmd in the same subdirectory, leaving only HTML format set

---
title: Subdir index
format: 
    html: default
---

5. create page.qmd in subdirectory, with no format setting in its YAML

---
title: Page in subdir
---

Expected behavior

Page-level YAML should override directory-level _metadata.yml such that if only one format is specified in file-level YAML, only that format should be rendered.

A more explicit solution (though not sure if consistent) might be to allow specific formats to be set to false in YAML headers and in _metadata.yml.

Actual behavior

Currently:

• file-level YAML correctly overrides format settings from site-level _quarto.yml
  • index.html in root correctly only has HTML format
  • about.html in root has both HTML and docx
• directory-level _metadata.yml correctly overrides _quarto.yaml formats in the relevant directory
  • page.html, with no format setting, has both formats

but:

• for a file in a subdirectory where the local _metadata.yml specifies multiple formats, all these formats are rendered for all files even if only one is specified in file-level YAML.
  • index.html in subdirectory has both formats, even though its YAML says to only have HTML

Your environment

• IDE: Version 2023.09.0+448 (2023.09.0+448)
• OS: MacOS Sonoma 14.0 (23A344)

Quarto check output

[✓] Checking versions of quarto binary dependencies...
Pandoc version 3.1.1: OK
Dart Sass version 1.55.0: OK
[✓] Checking versions of quarto dependencies......OK
[✓] Checking Quarto installation......OK
Version: 1.3.450
Path: /Applications/quarto/bin

[✓] Checking basic markdown render....OK

[✓] Checking Python 3 installation....OK
Version: 3.11.5
Path: /opt/homebrew/opt/python@3.11/bin/python3.11
Jupyter: 5.1.3
Kernels: python3, ir

(-) Checking Jupyter engine render....0.00s - Debugger warning: It seems that frozen modules are being used, which may
0.00s - make the debugger miss breakpoints. Please pass -Xfrozen_modules=off
0.00s - to python to disable frozen modules.
0.00s - Note: Debugging will proceed. Set PYDEVD_DISABLE_FILE_VALIDATION=1 to disable this validation.
[✓] Checking Jupyter engine render....OK

[✓] Checking R installation...........(None)

  Unable to locate an installed version of R.
  Install R from https://cloud.r-project.org/

[mcanouil]

This is not a bug I believe (or not at the level you think it is), since YAML are merged and in case of conflict, file level overwrites directory level which overwrites project level, see https://quarto.org/docs/projects/quarto-projects.html#metadata-merging.

You should look into project profiles, see https://quarto.org/docs/projects/profiles.html.

Note that you are using a deprecated command to create your project, see https://quarto.org/docs/websites/#quick-start.

[mcanouil]

@petrbouchal Would you mind sharing your example as a Git repository which fully reproduce the issue as currently somethings are missing such as how you render your project.

[petrbouchal]

Thanks. I put the example at https://github.com/petrbouchal/quarto-formats.

Unless I am missing something, what I see is that the logic you describe is not followed:

• page-level YAML overrides project-level YAML (_quarto.yml) where no directory level _metadata.yml file is present - as expected
• _metadata.yml overrides _quarto.yml for files in its directory - as expected
• but: page-level YAML does not override directory-level _metadata.yml in that same subdirectory, or at least not at the level of format enumeration - this is unexpected

I'll see if profiles can help here. This is one project with only one version, I just want to be able to turn off rendering of additional formats for an individual file in a directory, where for that directory different formats are rendered than for the entire project, like so:

1. in the whole project: render HTML only
2. except for directory A, where additionally docx should be rendered for all files but the index (for which only HTML is needed)

[cderv]

We do merge document level format meta with Directory level format meta, to create a unique list.

quarto-cli/src/command/render/render-contexts.ts

Lines 428 to 434 in 0efcbda

	// determine formats (treat dir format keys as part of 'input' format keys)
	let formats: string[] = [];
	const projFormatKeys = formatKeys(projMetadata);
	const dirFormatKeys = formatKeys(directoryMetadata);
	const inputFormatKeys = ld.uniq(
	formatKeys(inputMetadata).concat(dirFormatKeys),
	);

So document level metadata does filter project level one but not directory level metadata. It has been like this since the start I believe (looking back at file history).

It seems this is not as clear in our documentation at https://quarto.org/docs/projects/quarto-projects.html#shared-metadata

The one exception to metadata merging is format. If the document-level YAML defines a format, it must define the complete list of formats to be rendered.

@dragonstyle does this behavior sounds right to you ? Are should there be an override ?

Thanks.

[dragonstyle]

This behavior does sound correct. For projects, we special case this format behavior because it is common to want to have things like a PDF document or RevealJS presentation within a website which isn't also a web page. In the case of _metadata.yml, I think the appropriate metal model is that we union the metadata (with merging) - the directory level _metadata.yml file is a way to merge metadata into all of the documents in a directory.

[ar-puuk]

Hi, jumping in on this issue as I'm encountering (what I think is) the same issue with my Quarto website related to how document-level format options interact with _metadata.yaml.

I have a _metadata.yaml file in my posts folder that sets common format options for html, typst, and docx. This works well for blog posts, as they all share the same formatting.

However, I occasionally want to include a revealjs presentation alongside my posts. When I try to specify format: revealjs in the front matter of the presentation document, it appears to be overridden by the _metadata.yaml settings, which don't include revealjs. As a result, Quarto attempts to render the presentation as an html document and fails with the following error:

ERROR: NotFound: The system cannot find the file specified. (os error 2): rename 'D:\GitHub\ar-puuk.github.io\posts\quantifying-economic-impacts\index.html' -> 'D:\GitHub\ar-puuk.github.io\docs\posts\quantifying-economic-impacts\index.html'

Stack trace:
    at Object.renameSync (ext:deno_fs/30_fs.js:267:3)
    at safeMoveSync (file:///C:/Program Files/Quarto/bin/quarto.js:2354:14)
    at renderProject (file:///C:/Program Files/Quarto/bin/quarto.js:87372:17)
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async Command.actionHandler (file:///C:/Program Files/Quarto/bin/quarto.js:91975:32)
    at async Command.execute (file:///C:/Program Files/Quarto/bin/quarto.js:8253:13)
    at async Command.parseCommand (file:///C:/Program Files/Quarto/bin/quarto.js:8143:20)
    at async quarto (file:///C:/Program Files/Quarto/bin/quarto.js:133948:9)
    at async file:///C:/Program Files/Quarto/bin/quarto.js:133978:9
    at async file:///C:/Program Files/Quarto/bin/quarto.js:133830:20

This seems to happen because _metadata.yaml specifies a set of formats that ends up excluding revealjs, even if it's defined at the document level.

I believe it would be helpful if Quarto allowed document-level format declarations to override folder-level _metadata.yaml settings (in line with what is mentioned in the Quarto documentation) or at least provided a flag like format-override: true or format-override: merge.

This would prevent the need to duplicate format settings across dozens of documents. For example, I currently have a shared _metadata.yaml like this:

format:
  html:
    toc: true
    number-sections: true
    number-depth: 2
    html-math-method: katex
    code-link: true
    code-tools: true
  typst:
    toc: true
    number-sections: true
    number-depth: 2
  docx:
    toc: true
    number-sections: true
    number-depth: 2

But if I add a single Revealjs presentation, I’d need to move this format block into each individual post just to make one exception.

Unfortunately, I don’t know how to create a minimal reproducible example, but you can see my full repo here: https://github.com/ar-puuk/ar-puuk.github.io

Thanks for the great tool and your time looking into this!

[mcanouil]

@ar-puuk please open a new Q&A discussion and provide a small fully reproducible example (not your whole project).
Thanks!