Skip to content

Using uc option with proxy in multiprocessing repeats same ip #2101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
FranciscoPalomares opened this issue Sep 12, 2023 · 11 comments
Closed
Labels
self-resolved You closed/solved the issue on your own UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode

Comments

@FranciscoPalomares
Copy link

FranciscoPalomares commented Sep 12, 2023

image

Hi, using this code in multiprocessing and different proxy, repeats ip:
driver = Driver(uc=True, proxy=proxy_dict['http'], headless=headless, uc_cdp_events=api,incognito=True)


Thanks

@mdmintz mdmintz added self-resolved You closed/solved the issue on your own UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode labels Sep 12, 2023
@punkerpunker
Copy link

punkerpunker commented Sep 15, 2023

This does seem to still be an issue, I can take a look into if you haven't started yourself yet @mdmintz, thanks!

Though to me the problem is not only goes with multiprocessing, but also a regular headless2 mode doesn't seem to be working well with authenticated proxy. (ip-auth proxy goes well, so I guess the issue is related to the browser extension)

@mdmintz
Copy link
Member

mdmintz commented Sep 15, 2023

You have to enable that option for multiple proxies. See the documentation: #1832

  • --multi-proxy (Command-line option for pytest)
  • multi_proxy=True (SB manager, Driver manager, DriverContext manager)

@punkerpunker
Copy link

punkerpunker commented Sep 15, 2023

@mdmintz Is this something that needs to be enabled even when I use single authenticated proxy single-threaded? Thanks!

@mdmintz
Copy link
Member

mdmintz commented Sep 15, 2023

@punkerpunker If using a single proxy (either single-threaded or multi-threaded), you won't need the option.

Using --multi-proxy / multi_proxy=True will create a custom proxy config for every test that uses a proxy.

@punkerpunker
Copy link

punkerpunker commented Sep 15, 2023

Thanks @mdmintz!

What I am trying to communicate, is that after the upgrade to version 117, extension that's wrapping a user authenticated proxy doesn't work in headless2 mode.

I am not the owner of the original issue, felt it is related. I can open a new one.

@mdmintz
Copy link
Member

mdmintz commented Sep 15, 2023

@punkerpunker Are you using SeleniumBase 4.18.6? https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.18.6
Chrome 117 made changes to headless mode, and SeleniumBase was already patched for it.

@punkerpunker
Copy link

punkerpunker commented Sep 15, 2023

@punkerpunker Are you using SeleniumBase 4.18.6? https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.18.6
Chrome 117 made changes to headless mode, and SeleniumBase was already patched for it.

I just double checked it - yes, that's the case with the latest version of seleniumbase @mdmintz

@mdmintz
Copy link
Member

mdmintz commented Sep 15, 2023

@punkerpunker Can you be more specific on your issue? You mentioned several topics, but that you only had one issue.

Which of these most-closely resembles the issue you're experiencing?

  1. Proxy settings aren't being applied at all.
  2. Only one proxy is being used for all tests.
  3. An error message appears somewhere.
  4. Multi-threading isn't working.
  5. UC Mode is getting detected in headless mode.

If it's the last one, (5), check your user-agent, and see if HeadlessChrome is there. You can change your user-agent with an arg: agent. Make sure you set it to one that usesChrome, because HeadlessChrome gets you detected.

@mdmintz
Copy link
Member

mdmintz commented Sep 15, 2023

I found the issue, it's most-certainly number 5. The workaround right now is to specify the agent if using an authenticated proxy in headless mode. I'll probably create another ticket for this.

@mdmintz
Copy link
Member

mdmintz commented Sep 15, 2023

I opened another ticket for it: #2114
(It's off-topic from this original ticket.)

@mdmintz
Copy link
Member

mdmintz commented Sep 15, 2023

@punkerpunker - resolved in 4.18.7 - https://github.com/seleniumbase/SeleniumBase/releases/tag/v4.18.7

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
self-resolved You closed/solved the issue on your own UC Mode / CDP Mode Undetected Chromedriver Mode / CDP Mode
Projects
None yet
Development

No branches or pull requests

3 participants