For years, I had been seeing a problem with NetWare servers connected to bridged
Internet connections not being able to work with secondary IP addresses. The primary address was
fine, but you could not access any secondary IP address bound to the network card.
This affects people with bridged connections to the ISP, and people hooking up BorderManager
directly to a cable modem. It only affects servers with more than one public IP address assigned to
the server.
The problem itself is an ARP issue with how the NetWare TCP stack is designed to respond to ARP
requests. Novell TID 10077130 tells you that this is working by design, but it sure causes problems.
If you suspect that you have this problem, here is how you can confirm it and see what is going on:
1. SET TCP ARP DEBUG=ON
2. Look at the logger screen. Have someone on the Internet ping a secondary IP address assigned to
your server. You should be seeing ARPRECEIVE requests. Note that for requests to the secondary
IP addresses assigned to your server, there are no ARPSEND replies going out.
3. Look closely at the sending IP address of the ARP requestor, as well as the target IP address. If
the target IP address is the secondary address on the public side of the server, this is the data
of interest to you. The sending IP address will be of a server located at the ISP.
The problem occurs when the sending IP address is NOT in the same subnet as your public IP address
on the NetWare server. For instance, you have a 12.1.2.194 (255.255.255.240) IP address assigned to
your public NIC, giving you 13 usable IP addresses for your server, and your default gateway is
12.1.2.193. You would expect that ARP requests would be coming from 12.1.2.193, and if you had a
routed Internet connection, you would see ARP requests from there. But in this case you see them
from a completely different subnet - for example 10.40.0.208.
Per the Novell TID mentioned above, NetWare's TCP/IP stack will not respond to ARP requests coming
from another subnet. You see this in the ARP debug data showing up as ARPRECEIVE's that never have
an ARPSEND reply.
A Workaround
It took me years to think of this, as some nice simple ideas do sometimes, but it has worked for me.
Since the problem is that the server will not respond to requests from another subnet, we need to
make sure that the requests are seen to be coming from a local-attached subnet. AND we need respond
to requests on one subnet with ARP data for another one, which means using Proxy ARP. So we bind
another public IP address, in the same subnet as the sending router at the ISP, and enable Proxy
ARP on it. You have to be careful in case the router at the ISP changes addresses, especially if
it changes to the same one you assign to the server, but unless that happens this trick should work.
EXAMPLE: (taken from a real customer experience)
ARPRECEIVE coming from 10.40.0.208, for addresses on a 12.x.x.x (255.255.255.240) subnet.
1. In INETCFG, add another public binding 10.40.0.200 (255.255.255.0). (The ARP requester address
must be within this subnet).
2. In the new binding, go to Configure TCP/IP Bind Options, Expert TCP/IP Bind Options, ARP Options
3. Set Force Proxy ARP to Enabled
4. Reinitialize System and test. The secondary addresses should now work.
There are potential issues with this, mostly involving conflicts with internal subnets matching
whatever might be in use at the ISP. You could try using a 255.255.255.252 or .248 mask on the
new public subnet to limit the exposure, but in my experience, bridged connections to the Internet
tend to involve small clients with not many internal subnets. (And you could probably change
internal IP addresses if necessary to get around this issue).