next/image to use <picture> element for art direction support

[drudunn]

Rationale

next/image does not currently support multiple image sources, commonly used for specific art direction at different screen sizes, rendered as a picture element.

Each source element includes

1. a media value to target a specific media query, usually a screen-size range, and
2. a srcSet value to describe pixel density renditions or width values

The picture element will also usually describe alternate sources for complimentary alternate mime/type renditions.

Idea

I'm keen to collaborate on a PR for the solution, but here are some ideas for how it could look like for end-users—I appreciate each of these will require further discussion.

Image component

The Image component could have a new complimentary sources prop for specifying each source file

<Image
  src={'/tile.jpg'}
  sources={['/tile-sm.jpg', '/tile-med.jpg', '/tile-lrg.jpg', '/tile-xl.jpg']}
/>

The src prop could alternatively support an array of images

<Image
  src={['/tile.jpg', '/tile-sm.jpg', '/tile-med.jpg', '/tile-lrg.jpg', '/tile-xl.jpg']}
/>

Either option naively expects there is a source value for each media query in the sourceSizes definition in next.config

In lieu of variants specific for each media query, duplicate values can be provided to ensure the array is at the expected length—ideally type-safe somehow.

<Image
  src={['/banner.jpg', '/banner-med.jpg', '/banner-med.jpg', '/banner-xl.jpg', '/banner-xl.jpg']}
/>

Source definitions

source media queries could be managed in the next.config, to follow convention
These values would define the number of source elements and their min-width values respectively.

module.exports = {
  images: {
    sourceSizes: [450, 768, 1200, 2000],
  },
}

This definition would generate the related media queries, eg.

<picture>
  <source media="(min-width: 2000px)" srcset="tile-xl.jpg 1x,  tile-xl-2x.jpg 2x">
  <source media="(min-width: 1200px)" srcset="tile-lrg.jpg 1x, tile-lrg-2x.jpg 2x">
  <source media="(min-width: 768px)"  srcset="tile-med.jpg 1x, tile-med-2x.jpg 2x">
  <source media="(min-width: 450px)"  srcset="tile-sm.jpg 1x,  tile-sm-2x.jpg 2x">
  <img src="tile.jpg" width="400" height="300" alt="" />
</picture>

Whilst there could be some reuse with deviceSizes, the default values provided by Next would produce a verbose number of source elements if we used those to define the media queries.

Format defintions

Specific progressive mime/type renditions could also be managed in the next.config

module.exports = {
  images: {
    formats: ['avif', 'webp'],
  },
}

This definition would generate the related type variant for each media query source, eg.

<picture>
  <source type="image/avif" media="(min-width: 2000px)" srcset="tile-xl.jpg?fmt=avif 1x,  tile-xl-2x.jpg?fmt=avif 2x">
  <source type="image/webp" media="(min-width: 2000px)" srcset="tile-xl.jpg?fmt=webp 1x,  tile-xl-2x.jpg?fmt=webp 2x">
  <source media="(min-width: 2000px)" srcset="tile-xl.jpg 1x,  tile-xl-2x.jpg 2x">

  <source type="image/avif" media="(min-width: 1200px)" srcset="tile-lrg.jpg?fmt=avif 1x, tile-lrg-2x.jpg?fmt=avif 2x">
  <source type="image/webp" media="(min-width: 1200px)" srcset="tile-lrg.jpg?fmt=webp 1x, tile-lrg-2x.jpg?fmt=webp 2x">
  <source media="(min-width: 1200px)" srcset="tile-lrg.jpg 1x, tile-lrg-2x.jpg 2x">
  ...
</picture>

srcSet units

In my experience, each source element will most often include a srcSet to describe pixel density renditions, rather than width values, as the media queries generally drive the width variants.

I would expect the srcSet to use pixel density renditions by default, but ideally this would be an option in next.config

module.exports = {
  images: {
    sourceUnit: 'width', // default: 'density'
  },
}

---

Further discussion

width and height params for each source element

In my experience, art-directed renditions tend to have varied aspect-ratio values.

See here for a usage example, but each source can now be provided width and height attributes to aid reducing CLS.

How to best provide these optional values is up for discussion.

Media expressions

I have made an assumption that the media query definitions are range-based, using min-width declarations.

There may be instances where users may want to define max-width or defined ranges instead.
Ideally this would be an option in next.config

module.exports = {
  images: {
    sourceMediaPrefix: 'max', // default: 'min'
    sourceSizes: [450, 767, [768, 1200], 2000]
  },
}

The nested array value could be a way to define a range query, eg.

<source media="(min-width: 768px) and (max-width: 1200px)" srcset="tile-med.jpg 1x, tile-med-2x.jpg 2x">

Alternatively, string-based logical operators may be more suitable;

module.exports = {
  images: {
    sourceSizes: ['<450', '<767', ['>=768', '<1023'], '>=1024', '>2000'],
  },
}

There may also be instances where users want to provide more complex media queries targeting orientation or other features. I'm not sure how this would be supported.

Source range overrides

There may be instances where some images in your application either have more or less source renditions than the range defined in sourceSizes in next.config

We will likely require some prop-based override mechanism to aid these use cases.

Related

• Formats: Support <picture> and <source> in next/image #25393 discusses an improved format specification method that could complement this proposal.
• Media Density: Optimise images for high-density displays #26065 discusses using source elements to target high-density displays, which could complement or compete with this proposal
• Media prefers: this comment requests support for alternate images based on color-scheme preference, which could complement or compete with this proposal
• Similar requests: [next/image] Responsive images with "art direction" #19880, Different mobile and desktop images with Next.js image #21379 and Next.js + styled components -> @media breakpoint + change image content ? #26625 could be solved by this proposal

---

I have had to implement complex CMS-driven picture elements a few times in the wild now and am keen to collaborate on a PR for the solution.

I look forward to any feedback or suggestions!

[drudunn]

Having discussed some of my thinking and trade-offs with @atomicleopard, there is an alternate suggestion for sourceSizes that is worth discussing its benefits and trade-offs.

Source definitions

An alternate solution could be to define a sources object in next.config
Creating named options to use:

module.exports : {
  images: {
    sources: {
      small: '(min-width: 450px)'
      medium: '(min-width: 768px)',
      large: '(min-width: 1200px)',
      xl: '(min-width: 2000px)',
    }
  },
}

The props you pass could then be a matching object

<Image
  src={'/tile.jpg'}
  sources={{
    small: '/tile-sm.jpg',
    medium: '/tile-med.jpg',
    large: '/tile-lrg.jpg',
    xl: '/tile-xl.jpg'
  }}
/>

Flexibility

I originally opted for plain number values in the sourceSizes config above, as it makes guarding that appropriate values are provided easier.

Having it more open as strings would make this trickier but not impossible.
This could also solve for the other media query options however:

module.exports : {
  images: {
    sources: {
      tablet: '(min-width: 450px)'
      desktop: '(min-width: 1200px)',
      darkMode: 'prefers-color-scheme: dark',
      highDensity: '(-webkit-min-device-pixel-ratio: 1.5)',
    }
  },
}

could then be used as

<Image
  src={'/tile.jpg'}
  sources={{
    tablet: '/tile-tablet.jpg',
    desktop: '/tile-desktop.jpg',
    darkMode: '/tile-dark.jpg',
    highDensity: '/tile-hd.jpg'
  }}
/>

Omitted values

With this suggestion, there was an expectation that if values were omitted during usage, they would not be included in the rendered output.

<Image
  src={'/tile.jpg'}
  sources={{
    desktop: '/tile-desktop.jpg',
    darkMode: '/tile-dark.jpg'
  }}
/>

This adds great flexibility but equal complexity.

Benefits

I previously noted

Source range overrides

There may be instances where some images in your application either have more or less source renditions than the range defined in sourceSizes in next.config

We will likely require some prop-based override mechanism to aid these use cases.

This named object approach makes a prop-based override simple, and ensures in instances where you want less or more source elements in your picture, its up to the developer at time of use.

<picture>
  <source media="(min-width: 1200px)" srcset="tile-desktop.jpg 1x, tile-desktop-2x.jpg 2x">
  <source media="prefers-color-scheme: dark" srcset="tile-dark.jpg 1x,  tile-dark-2x.jpg 2x">
  <img src="tile.jpg" width="400" height="300" alt="" />
</picture>

Trade-offs

The reason I used a fixed set of values was to ensure that there were appropriate renditions used for each screen size, as the media query driven sources are what provides the responsive options (at least in my default setup above).

Using my example above, if no tablet source was provided we would end up serving an image too small (or too large), to tablet screens.

<picture>
  <source media="(min-width: 2000px)" srcset="tile-xl.jpg 1x,  tile-xl-2x.jpg 2x">
  <source media="(min-width: 1200px)" srcset="tile-lrg.jpg 1x, tile-lrg-2x.jpg 2x">
- <source media="(min-width: 768px)"  srcset="tile-med.jpg 1x, tile-med-2x.jpg 2x">
  <source media="(min-width: 450px)"  srcset="tile-sm.jpg 1x,  tile-sm-2x.jpg 2x">
  <img src="tile.jpg" width="400" height="300" alt="" />
</picture>

This is because the media query is what drives the size options in the picture element, as the srcSet is used to describe density options, rather than widths.

In this scenario, to fill the missing <source media="(min-width: 768px)" we'd have to use one of the other source files for that query range, and rely on the CDN's on-demand rendition for a custom width. Next would have to make an executive call on the priority order of which source to use when one or many are missing.

<picture>
  <source media="(min-width: 2000px)" srcset="tile-xl.jpg 1x,  tile-xl-2x.jpg 2x">
  <source media="(min-width: 1200px)" srcset="tile-lrg.jpg 1x, tile-lrg-2x.jpg 2x">
+ <source media="(min-width: 768px)"  srcset="tile-lrg.jpg?w=768 1x, tile-lrg.jpg 2x">
  <source media="(min-width: 450px)"  srcset="tile-sm.jpg 1x,  tile-sm-2x.jpg 2x">
  <img src="tile.jpg" width="400" height="300" alt="" />
</picture>

We could define a priority pattern, but it would be very opinionated.

Priority pattern

This is something we currently do for one of our projects. The pattern we use is as follows:

source	priority
large	large or medium or null
medium	medium or large or null
small	small or medium or large
img (fallback)	small or medium or large

This pattern ensures a source is provided when there are enough sources defined to warrant rendering a picture element.

In the scenario where neither large or medium sizes are provided, we default to rendering an image with srcset and sizes like next/image currently generates.

This priority pattern could also be a next.config option though:

module.exports : {
  images: {
  sources: {
    small: {
      media: '(min-width: 450px)',
      priority: ['small', 'medium', 'large']
    },
    medium: {
      media: '(min-width: 768px)',
      priority: ['medium', 'large']
    },
    large: {
      media: '(min-width: 1200px)',
      priority: ['large', 'medium']
    },
    fallback: { // for `img` src value
      priority: ['default', 'small', 'medium', 'large'] // default = `src` prop value, but could be potentially omitted 
    }
  }
}

---

Other people's mileage may vary. Whilst these decisions are driven by experiences across projects, this may be unique in the grand scheme of things.

I'd be interested in any feedback the team has.

[leerob]

Wow, thank you for the amount of detail and effort that went into this. Incredibly useful! 🥳

There are currently two other related discussions around picture:

• Support <picture> and <source> in next/image #25393
• Optimise images for high-density displays #26065

The team is interested in exploring this more and @atcastle will be leading that work.

[drudunn]

Thanks, would be glad to help out if the team would like.

Also have a couple of projects that could be of use to help validate the implementation.

[leerob]

Update: We have extracted the core logic from next/image into a new unstable_getImgProps() function.

This allows usage outside of <Image>, such as:

1. Working with background-image or image-set
2. Working with canvas context.drawImage() or simply new Image()
3. Working with <picture> media queries to implement Art Direction or Light/Dark Mode images

Example

import { unstable_getImgProps as getImgProps } from 'next/image'

export default function Page() {
  const common = { alt: 'Hero', width: 800, height: 400 }
  const { props: { srcSet: dark } } = getImgProps({ ...common, src: '/dark.png' })
  const { props: { srcSet: light, ...rest } } = getImgProps({ ...common, src: '/light.png' })

  return (<picture>
  <source media="(prefers-color-scheme: dark)" srcSet={dark} />
  <source media="(prefers-color-scheme: light)" srcSet={light} />
  <img {...rest} />
</picture>)
}

PR: #51205

[Adamkaram]

it dosen't work with me

import { unstable_getImgProps as getImgProps } from 'next/image'


export function Logo() {

  const common = { alt: 'logo', width: 40, height: 30 }
  const { props: { srcSet: dark  } } = getImgProps({ ...common, src: '/images/logo/logo.webp' })
  const { props: { srcSet: light , ...rest } } = getImgProps({ ...common, src: '/images/logo/logo-2.webp' })
 


  return (
    <picture>

      <source
       
       media="(prefers-color-scheme: dark)"
        srcSet={dark} />
  <source media="(prefers-color-scheme: light )" srcSet={light} />
  <img  {...rest} />
    </picture>
  )
}