Embrace more network chips (W5200 and W5500 at least) to be more generic

[q2dg]

My main reasoning is here (arduino/Arduino#5170) but it was centered about web documentation, so I think maybe here is the right place to explain the issue. Let's go:

None of current official Arduino products has Ethernet conectivity (sadly). So...why keep maintaining this library? If the plan is to be not tied to any specific official Arduino product, then I think this library should be more generic (as it is this: https://github.com/Wiznet/WIZ_Ethernet_Library, at least). Current situation is no sustainable because W5100 is vanishing little by little and currently this library doesn't offer other alternative. It's the same case as TFT library...if it were generic, maybe someone used it (as it is used LiquidCrystal library, for instance) but being tied to an specific chip, i_f this chip is not associated to an official product,_ it's a dead-end.

Another solution would be offer a new shining product: an Ethernet shield with TLS integrated chip and SD card , (and therefore, modify this library accordingly)...but I suspect this is not the plan

[njh]

See also: https://github.com/adafruit/Ethernet2

[q2dg]

Well, in fact, now THE Arduino Ethernet shield is the former .org's Arduino Ethernet 2, based on W5500 (https://www.arduino.cc/en/Main/ArduinoEthernetShield) so...I think official library should be updated accordingly. But it's just an opinion.

EDIT: In fact, https://www.arduino.cc/en/Reference/EthernetV2 returns nothing

[PaulStoffregen]

Please please please do not just use Arduino.org's Ethernet2 library. It's based on a very old copy of Ethernet, from before SPI transactions.

It's not difficult to make an Ethernet library which automatically detects the chip and works with all 3 SPI-interface Wiznet parts. My own fork of Ethernet has long auto-detected W5100 & W5200, and just recently I added support for W5500.

https://github.com/PaulStoffregen/Ethernet

My fork also has quite a lot of optimization work in the socket layer. It runs much faster, but this does cost some extra RAM usage. So perhaps my fork would be best to use on the ARM boards like Zero & Due, and another fork with the original socket layer could be used on AVR?

Just yesterday, only hours after finally getting my W5500 code to work reliably, I (re)discovered an auto-detecting contribution has been pending for quite a long time.

arduino/Arduino#2325

Normally I don't comment on these sorts of "Arduino ought to do XYZ" threads, especially when it's about hardware product choices. But please, for the sake of so many Arduino users who will use this shield together with other libraries, I beg of you, please please please don't publish a sloppy least-possible-effort hack Ethernet2. My fork or that old contribution can make everything "just work" automatically for everyone on all Wiznet-based shields, and preserving the SPI transaction support will spare a lot of users who try to combine Ethernet with SPI-based libs using interrupts from terribly painful frustration.

[njh]

I was completely unaware of your work Paul, and I suspect most other Arduino users will be too. I have not tried it myself but it sounds like a real improvement.

I'm personally unaware of how the official Arduino development model works and how something becomes official/supported. I don't quite understand how there can be an Arduino Ethernet Shield being sold in high street stores around the world, but without proper library support.

Unfortunately at the moment, the @adafruit repo is the most official W5500 library out there. And I suspect that only came into existence because of the lack of official W5500 support from Arduino.

[PaulStoffregen]

I was also completely unaware Arduino.cc was adopting Arduino.org's W5500-based shield. Guess it makes sense now that they're starting to settle, but still news to me.

As a general rule, I publish my work on github, but I don't "evangelize" too much. That's usually just a recipe for frustration.

[q2dg]

I think this discussion deserves a lot more visibility. Maybe in developers mail list, don't you think?

In general, the degree of support of "new kids in town" (as well as "returned kids" like Due or old Wifi Shield -wtf??-) is a real mistery.

I'd like to know some roadmap (in regards to hardware stock and support) because I'm starting to be very dazzed, confused and tired. Tje worst part is I suspect there's no such roadmap.

[PaulStoffregen]

I think this discussion deserves a lot more visibility. Maybe in developers mail list, don't you think?

I'd suggest patience. At the very least wait until next week, since today is a major USA holiday. When/if you do post, perhaps take a gentle tone, avoid the instance you've previously had about Arduino's hardware product decisions, and focus on ease-of-use benefits for users.

[PaulStoffregen]

I've been thinking about this a bit over the last couple days.

Instead of asking the Arduino devs to put their own time and effort into evaluating these many Ethernet library changes, perhaps I should clean up my version meant for Teensy and retest it on various non-Teensy boards, and then publish with a distinct name.

Then you (and perhaps others) could suggest to people on the Arduino forums who are having issues with Ethernet to try the optimized, works-with-all-chips version. Ideally, they could be encouraged to leave feedback somewhere, even if only "me too".

After it's been used with good results by hundreds or even dozens of real people across the many different types of modern Arduino hardware, then we could make a compelling argument to the Arduino devs to just replace the old Ethernet. They could dig into the code and technical details if they like, but offering them a well tested replacement would probably work much better than asking them to merge many different patches and improvements, only to then need to begin testing.

[njh]

That sounds like a very good idea @PaulStoffregen.

Would also be good to get it into the Arduino library manager with its distinct name.

[PaulStoffregen]

Yes, will soon (next couple weeks) submit distinctly named version for the library manager.

Right now I'm (again) working on trying to figure out why all these libraries are so very slow. On WebClient, Ethernet2 achieves about 10 kbytes/sec. My optimizations speed it up to 95 kbytes/sec. But that's still pretty terrible. In theory, about 1 Mbyte/sec ought to be possible with 12 or 14 MHz SPI clock and W5500's protocol overhead.

[PaulStoffregen]

I see you just posted on the developers mail list. Apparently patience is difficult. Would have been better to wait another week or two.

[PaulStoffregen]

I'm still looking into why W5200 and W5500 are so much slower when allocating larger per-socket buffers. I purchased a network tap, so I can better see what's really happening. It's supposed to arrive early next week.