[nSLUG] ethernet and the modern linux

George N. White III gnwiii at gmail.com
Sun Sep 28 14:06:19 ADT 2008


On Sat, Sep 27, 2008 at 10:54 PM, Stephen Gregory <nslug at kernelpanic.ca> wrote:

> George N. White III wrote:
>
>> 1.  autonegotiation considered harmful
>
> get better Ethernet cards. 10 years ago autonegotiation sucked because
> every vendor had a different proprietary method to do it. Are you still
> seriously having issues with recent hardware? There is all manner of
> junk hardware in the lab and the only issue we have is that the 100FX
> fibre cards don't negotiate.

I'm considering that.   Any recommendations for good 5v PCI interfaces
that would work in a P-III?  Actually, I'd love to dump the P-III's and get
current gear, but we use some special interface cards that need an
ISA bus.  Has anybody used those bus extender boxes that add ISA
slots to a PCI bus?

One of the ugly secrets about "rocket science" is that it relies on really
ancient hardware because there are long delays from initial design to
launch.   Much of the software is written pre-launch and if you are lucky
might get a major revision once the satellite has demonstrated a degree
of reliability (too many die young).  Many important satellites are still
operational 10 years after launch.  My brother worked at a lab that
designed a series of satellites and supervised their manufacture by a
contractor. Failure rates in the testing of early models were high, then
dropped off as glitches were ironed out.  After 10-15 years, however,
failure rates skyrocketed because the IBM mainframe that controlled
the tests was no longer able to stay up long enough to complete the tests.

Many NASA science programs have taken bug cuts.  The group that develops
software we use has had to severely limit the hardware platforms they
support.  They never made it to IRIX64, and are currently stuck at Solaris 8,
but use Fedora and MacOSX (the latter because the project must "publish or
perish", and Adobe apps are essential to high quality color printing (publishers
say "send us CMYK EPS files, but what they actually mean is send us files we
can open in Adobe Photoshop to do color separations, trapping, etc. )


>> 2.  udev considered harmful
>
> The problems you describe are problems with network-manager. N-M stinks
> and does weird things.

User tools to manage network connections are like mail user agents -- they
all suck, but N-M recent versions suck less.   You should see the messes
Windows users get after their laptops have been configured to use the network at
some institute they were visiting -- generally IT support wipes the systems and
reinstalls.

> UDEV mostly just creates entries in /dev. There is some great power
> behind UDEV and it should be studied instead of cursed. Many devices in
> a modern computer can be hot-plugged. UDEV manages the naming and
> creation of /dev entries in a sane manner. The cursing should be
> reserved for the poor documentation.

Udev is a big help for those hot-plug devices, but not for networking on a
system that is "permanently" connected to the same port and should
stick with the boot configuration.

>> In the good old days with unix, you could force the settings by
>> building a custom kernel.
>
> You can still force everything. e.g in Debian N-M will not touch an
> interface if there is an entry in /etc/network/interfaces. Everything in
> UDEV can be customized in plain text config files.

I'm not sure what my debian (unstable) system, a Dell Optiplex GX1 PIII with
3com is doing.   I tried adding a file in modprobe.d to force full-100 on the
3c59x module.    It seems to be working if I manually remove the module and
then load it, but on boot dmesg has something to the effect that
"autoneg failed,
using half-duplex".  Eventually the system connected with full-100
(probably when
/etc/init.d/networking was run -- ntpd was able to connect to a
server).  It hasn't
been long enough to see if that system still does "autoneg failed,
using half-duplex".

>> Until recently, you could pass
>> options to control the settings when the module was loaded.
>
> Which distro? In Debian you can still control how and when modules are
> loaded, and which options are used. You can also compile a module free
> kernel.

Based on the tests I did Friday, the 3c59x module gets loaded by something
(udev) without using the options from modprobe.d, and only later is loaded
using the options.

>>   You can
>> also use ethtool to
>> make changes after the module is loaded, but these changes may be lost
>> after a network
>> "event".
>
> This is likely a driver issue or possibly a network-manager issue. UDEV
> only gets called when the device is loaded. If the device is getting
> loaded and unloaded there is something more seriously amiss.

I assume it is related to the recent changes made to support N-M behaviour
where if you unplug the ethernet from a laptop it will look for wireless.  The
system in question doesn't run a GUI, if it would stay on the network like it
is supposed to it would be managed from my office.   Next week I'll look at
the interfaces file (I remember editing one of those in days past, but don't
recall when or where).

>> 3.  full duplex considered harmful
>
> This a just don't understand. I can saturate a 100BaseT half-duplex
> network without trying. I can do this using boring hardware. If I were
> to drop my little 20 user network to half-duplex there would be an angry
> mob within minutes.

<http://www.sun.com/blueprints/0704/817-7526.pdf> says

 "Autonegotiation on the link is exchanged when:

  *  Link is initially connected
  *  Device at either end of the link is powered up
  *  Device is reset or initialized
  *  Renegotiation request is made"

We have problems with PC's being powered up, and after that I don't
know which of the other reasons applies.

The Sun document also states: "There are no requirements in the
standard to support locked down or forced configurations using
autonegotiation disabled. As a result, there are no requirements
for vendors to test multivendor interoperability between products
with autonegotiation disabled."

I gues this lets linux off the hook for not really supporting "autonegotiation
disabled", but since HD is what you get when autoneg fails, that is what
you have to use until you can get at least one of your vendors (switch or
interface) to admit they are noncompliant.

>> My view is that shit happens, and we should configure
>> systems to be robust.
>
> I agree. That is why all the ports on the switch at work are set to auto
> duplex and auto speed.
>
>
>> Apple,
>> who have lots of experience with a tightly controlled list of network
>> interfaces, recommend settings switches to force half-duplex,
>
> ... which says more about Apple's lack of ability then anything else.
>
>
>
> I recall
>> some discussions where it was claimed that full duplex was a fraud
>
> 10 years ago this argument had some merit. But not today. The 1GHz P3
> laptop in the basement doesn't seem to have any trouble filling 100BaseT
> full duplex.

We have three systems with gigabit interfaces.  The one with problems is
the MacPro, which has two interfaces.  One guess is that it blacklists the port
after an autoneg fail.   Again, manual intervention is needed to
restore the port.

> Half-duplex ethernet results in collisions. Collisions cause more work
> for the servers, workstations, and switches. Collisions cause the
> network to stop. In many cases this means the server stops. Even if a
> computer cannot fill 1Gb/sec ethernet is no reason to run in
> half-duplex. The point of full duplex is not that it is faster, but that
> there are no collisions

It isn't clear that collisions are a problem for switched ethernet.  What is
clear is that forcing HD and just allowing autoneg to fail keeps data flowing
without manual intervention.  In the long run the systems question need
to be replaced, but that depends on what the vendor (one system is
dedicated to a particular commercial app)  will support.   The other
system is used for UPS monitoring, where data volumes are very low
but and you want to know that it will work when the network is under
stress.

-- 
George N. White III <aa056 at chebucto.ns.ca>
Head of St. Margarets Bay, Nova Scotia



More information about the nSLUG mailing list