Image pack locations different between machines

[aldent95]

I'm currently working on converting one of my older websites to using just Webpacker and no Sprokets or old asset pipeline, but I've come across a weird issue that I can't seem to resolve.

I've followed guides as best I can for doing my images, and everything was working fine with the image_pack_tag for my .html.erb files on my local environment. I.E. <%= image_pack_tag 'media/slider/slider-1.jpg', class: 'responsive-img' %>

And the images being used are stored as below

As I said, this works fine on my local windows machine, both Dev, Prod and Prod assets:precompile

However when I push my code to Gitlab and have it built on a Linux VM, the pack paths change.
On my local I get the following in the manifest

  "media/slider/slider-1.jpg": "/packs/media/slider/slider-1-766084b0.jpg",
  "media/slider/slider-2.jpg": "/packs/media/slider/slider-2-49b9ae0c.jpg",
  "media/slider/slider-3.jpg": "/packs/media/slider/slider-3-32360107.jpg",
  "media/slider/slider-4.jpg": "/packs/media/slider/slider-4-6e14e879.jpg",
  "media/slider/slider-5.jpg": "/packs/media/slider/slider-5-e84f0a70.jpg",

But on the Linux build for prod I get the following

"media/images/slider/slider-5.jpg": "/packs/media/images/slider/slider-5-e84f0a7087263e5ba7f897dccc75ed98.jpg",
"media/images/slider/slider-4.jpg": "/packs/media/images/slider/slider-4-6e14e87967b11680d18c07ada4dc802a.jpg",
"media/images/slider/slider-3.jpg": "/packs/media/images/slider/slider-3-323601072f43c7e6dfbd3f668a29de9b.jpg",
"media/images/slider/slider-2.jpg": "/packs/media/images/slider/slider-2-49b9ae0ccfe0c9ea411bf0b56ee7fec6.jpg",
"media/images/slider/slider-1.jpg": "/packs/media/images/slider/slider-1-766084b0f3f37b461846e91b9c953d17.jpg",

It has the extra /images/ in it which is causing issues with all my tags in my code as they no longer point to the right files.

I've done everything I can think off to try and fix this but I'm not making any progress. Originally I thought it was a Dev vs Prod config issue but the prod manifest on my local machine is the same as the dev manifest. It's only once Gitlab builds it that it ends up different.

If anyone has any ideas I'd gladly try them as this is driving me mad.

[pedrofurtado]

Hey @aldent95 ! 👋 Thanks for report!

What version of webpacker you are using?

Anyway, can you try these examples below? Maybe the path in image_pack_tag can be defined as below:

<%= image_pack_tag 'media/images/slider/slider-1.jpg' %>

# or

<img src="<%= asset_pack_path('media/images/slider/slider-1.jpg') %>">

The prefix media/images/ is important, because webpacker "compile/exports" images using this prefix in path, and it is stored in manifest.json mapping.

Let me know your results in tests, using webpacker v5.2.1 (latest stable version) 🤝

[aldent95]

Whoops. I had a feeling I was forgetting something.

Ruby 2.7.1
Rails 6.0.3.4
Webpacker gem 5.2.1
@rails/webpacker 5.2.1
webpack-dev-server 3.11.2

Those two different version would work for the Gitlab prod build, but then it completely breaks my local build for doing development as Webpacker doesn't put them in media/images/ for some reason on my local machine, only media/

[pedrofurtado]

This is a really strange scenario. A better approach to help you, @aldent95 , is to create a reproducible repo (mini rails app) that has this error inside it. Can you create this repo for us? 🤝

Because I really don't know more details, with a reproducible repo will be more easy to debug 🕵️

[aldent95]

@pedrofurtado Done
You should be able to access it here: https://gitlab.com/aldent/imagesissue
If not let me know
Made it as minimal as I could.

That repo is doing the exact same thing as my project. anything not under the root images directory ends up in media/folder/file.ext on my local by media/images/folder/file.ext on my Gitlab build.

It's worth noting that images under the root image directory (In this case plane.png in the repo I've provided) work fine as media/images/plane.png on both my local and the Gitlab build.

[pedrofurtado]

@aldent95 Thank you so much for your time providing a reproducible repo 🤝

After I cloned your repo and executed rails server, appeared the message as described by you.

The origin of error is that media/slider/slider-1.jpg is a wrong path for webpacker, the gem cannot resolve this path.

To fix it, you just have to change media/slider/slider-1.jpg to media/images/slider/slider-1.jpg. The prefix media/images/ is required for this kind of file (images, in this case) 👍

After this change, the page showed correctly with the 2 images (a big red airplan and a green icon of airplan).

To test webpacker compilation in production/development/test mode, do the following:

• Ensure that you have config/master.key file
• Open config/webpacker.yml, and change all ocurrences of compile: true to compile: false
• Stop rails server
• Delete the whole folder public/packs/
• And then execute:
  • RAILS_ENV=production NODE_ENV=production rails webpacker:compile && RAILS_ENV=production NODE_ENV=production rails server
  • or
  • RAILS_ENV=development NODE_ENV=development rails webpacker:compile && RAILS_ENV=development NODE_ENV=development rails server
  • or
  • RAILS_ENV=test NODE_ENV=test rails webpacker:compile && RAILS_ENV=test NODE_ENV=test rails server
• After completion of commands you will see the two images in browser 🌟

Important note: the env variable RAILS_ENV/NODE_ENV must be the same in webpacker compilation and in application boot.

This command, for example, generates a error, because RAILS_ENV in webpacker compilation and in application boot not matches:

RAILS_ENV=test NODE_ENV=test rails webpacker:compile && RAILS_ENV=production NODE_ENV=production rails server

Webpacker compilation in test mode outputs the files in packs-test folder, and application executing in production/development mode will search files considering packs folder. If you want to change the ouput folder, you change this line here: https://gitlab.com/aldent/imagesissue/-/blob/master/config/webpacker.yml#L83

Did this resolution help you? If the error persists, even with these instructions, let us know here 🤝 👍

[aldent95]

Ok. So for some reason you don't get the same issue I did @pedrofurtado
I would assume, based on that at the fact that the issue only appears on my Windows dev machine, and that I assume you are probably on Linux just like my Gitlab build is, that the issue is specifically Windows related.

So that got me thinking that it's probably an issue with Windows vs Linux directory pathing.

Which lead me to search the webpacker source code for where the media/ part is set.
Which lead me to package/rules/file.js and specifically this block of code

name(file) {
          if (file.includes(sourcePath)) {
            return 'media/[path][name]-[hash].[ext]'
          }
          return 'media/[folder]/[name]-[hash:8].[ext]'
        },

I decided to debug this bit of code by adding some console outputs as follows

name(file) {
          console.log(file);
          console.log(sourcePath);
          if (file.includes(sourcePath)) {
            return 'media/[path][name]-[hash].[ext]'
          }
          return 'media/[folder]/[name]-[hash:8].[ext]'
        },

This lead me to the following discovery. For any given image in my source code I get the following using one file as an example.
file = D:\Dev\va-system\app\javascript\images\admin_images\acars.jpg
sourcePath = app/javascript

See how the slash directions don't match? I'd bet $10 that the slashes are in the opposite direction for the file variable on Linux environments.

Following the code snippit above, we can see that return 'media/[folder]/[name]-[hash:8].[ext]' will be executed as file.includes(sourcePath) will return false due to the different slash directions.

This results in media/admin_images/acars-901778fb.jpg being the outputted path for that specific image on my Windows machine when running rails webpacker:compile

When I changed the code in file.js to the following:

name(file) {
          return 'media/[path][name]-[hash].[ext]'
        },

rails webpacker:compile produced the expected output of the file which is media/images/admin_images/acars-901778fb545e56e0638cca924bc79913.jpg which matches the Linux output.

The joys of doing my development on a Windows machine. Not the first time I've come across silly things with Windows files and directories like this.

I spent some time upgrading my webpacker install so my configs worked with v6.0.0.beta.3 and that seems to already resolve the issue. My images on Windows are outputting to media/images/acars.jpg with that beta version.

I'm going to consider this solved at this point as Ill keep using the v6.0.0.beta.3 for now since that seems to solve the issue.
Ill keep this ticket open for now though in-case you want to investigate further for older versions, feel free to close if if you are happy with v6.0.0.beta.3 resolving the problem.

[pedrofurtado]

Hey @aldent95 ! 👋

Thanks for feedback and details, they were very helpful. Now I really understand the situation 😄

Can you try edit package/rules/file.js in order to be like this below? (using normalize function to be cross-platform):

const { join, normalize } = require('path') // CHANGE IS HERE
const { source_path: sourcePath, static_assets_extensions: fileExtensions } = require('../config')

module.exports = {
  test: new RegExp(`(${fileExtensions.join('|')})$`, 'i'),
  use: [
    {
      loader: 'file-loader',
      options: {
        name(file) {
          if (file.includes(normalize(sourcePath))) { // CHANGE IS HERE
            return 'media/[path][name]-[hash].[ext]'
          }
          return 'media/[folder]/[name]-[hash:8].[ext]'
        },
        esModule: false,
        context: join(sourcePath)
      }
    }
  ]
}

This code snippet above come from this PR: #2730

The PR fixed a problem related to paths in Windows platform, normalizing it, but it is available only in 6.x versions.

Let me know if the error disappears once adding the normalize function. If so, we can check the possibility to backport it in 5.x versions and release a minor version with this fix.

[aldent95]

@pedrofurtado Yep that did the trick. Both files in the test repo now start with media/images after making the change to file.js

[pedrofurtado]

@aldent95 That's a good news! 🎉

So, I will provide a PR to include this fix in 5.x stable versions 🤝

Thanks, @aldent95! 🤝

[pedrofurtado]

I think we can close this issue, @aldent95 .

The PR was merged, so now we just need to wait for a official release of 5.x

[rossta]

@pedrofurtado Thanks!