On 2012-01-25 19:03, James Carlson wrote:
Robin Axelsson wrote:
On 2012-01-24 21:59, James Carlson wrote:
Well, unless you get into playing tricks with IP Filter. And if you do
that, then you're in a much deeper world of hurt, at least in terms of
performance.
Here's what the virtualbox manul says about bridged networking:
"*Bridged networking*:
This is for more advanced networking needs such as network simulations
and running servers in a guest. When enabled, VirtualBox connects to one
of your installed network cards and exchanges network packets directly,
circumventing your host operating system's network stack.
Note what it says above. It says nothing about plumbing that interface
for IP on the host operating system.
I'm suggesting that you should _not_ do that, because you (apparently)
want to have separate interfaces for both host and the VirtualBox guests.
If that's not what you want, then I think you should clarify.
Perhaps the right answer is to put the host and guests on different
subnets, so that you have two interfaces with different subnets
configured on the same physical network. That can have some risks with
respect to multicast, but at least it works far better than duplicating
a subnet.
I suspect that the right answer is to plumb only *ONE* of them in the
zone, and then use the other by name inside the VM when creating the
virtual hub. That second interface should not be plumbed or configured
to use IP inside the regular OpenIndiana environment. That way, you'll
have two independent paths to the network.
Perhaps the way to do it is to create a dedicated jail/zone for
VIrtualBox to run in and "plumb the e1000g2" to that zone. I'm a little
curious as to how this would affect the performance I'm not sure if you
have to split up the CPU cores etc between zones or if that is taken
care of as the zones pretty much share the same kernel (and its task
scheduler).
I'm confused. If VirtualBox is just going to talk to the physical
interface itself, why is plumbing IP necessary at all? It shouldn't be
needed.
Maybe I'm the one being confused here. I just believed that the IP must
be visible to the host for VirtualBox to be able to find the interface
in first place but maybe that is not the case. When choosing an adapter
for bridged networking on my system, the drop-down menu will give me the
options e1000g1, e1000g2 and rge0. So I'm not sure how or what part of
the system that gives the physical interfaces those names. I mean if the
host can't see those interfaces how will VirtualBox be able to see them?
At least that was my reasoning behind it.
It's possible to have DHCP generate multiple addresses per interface.
And it's possible to use IPMP with just one IP address per interface (in
fact, you can use it with as little as one IP address per *group*). And
it's possible to configure an IPMP group with some static addresses and
some DHCP.
In order to make DHCP generate more IP addresses I guess I have to
generate a few (virtual) MAC addresses. Maybe ifconfig hadles this
internally.
You don't have to work that hard. You can configure individual IPv4
interfaces to use DHCP, and the system will automatically generate a
random DHCPv4 "ClientID" value for those interfaces.
For example, you can do this:
ifconfig e1000g0:1 plumb
ifconfig e1000g0:2 plumb
ifconfig e1000g0:1 dhcp
ifconfig e1000g0:2 dhcp
Using the old-style configuration interfaces, you can do "touch
/etc/dhcp.e1000g0:1" to set the system to plumb up and run DHCP on
e1000g0:1.
There's probably a way to do this with ipadm, but I'm too lazy to read
the man page for it. I suggest it, though, as a worthwhile thing to do
on a lazy Sunday afternoon.
I'll look into it if "all else fails". I see that the manual entry for
ipadm is missing in OI. I will also see if there is more up-to-date
documentation on the ipmp. I assume that when a "ClientID" value is
generated a MAC address also comes with it, at least when it negotiates
with the DHCP server.
But those are just two small ways in which multiple interfaces
configured in this manner are a Bad Thing. A more fundamental issue is
that it was just never designed to be used that way, and if you do so,
you're a test pilot.
This was very interesting and insightful. I've always wondered how
Windows tell the difference between two network connections in a
machine, now I see that it doesn't. Sometimes this can get corrupted in
Windows and sever the internet connection completely. If I understand
correctly, the TCP stack in Windows is borrowed from Sun. I guess this
is a little OT, it's just a reflection.
No, I don't think they're related in any significant way. The TCP/IP
stack that Sun acquired long, long ago came from Mentat, and greatly
modified since then. I suspect that Windows derives from of the BSD
code, but I don't have access to the Windows internals to make sure.
In any event, they all come from the basic constraints of the protocol
design itself, particularly RFC 791, and the "weak ES" model.
I'm sure a good book in computer networking would give a proper insight
into that...
I will follow these instructions if I choose to configure IPMP:
http://www.sunsolarisadmin.com/networking/configure-ipmp-load-balancing-resilience-in-sun-solaris/
Wow, that's old. You might want to dig up something a little more
modern. Before OpenIndiana branched off of OpenSolaris (or before
Oracle slammed the door shut), a lot of work went into IPMP to make it
much more flexible.
I'll see if there is something more up-to-date. There are no man entries
for 'ipmp' in OI and 'apropos' doesn't work for me.
Try "man ifconfig", "man ipmpstat", "man if_mpadm". Those should be
reasonable starting points.
Thanks, these man pages exists. I saw in the ifconfig that there is some
info about ipmp although it is brief.
In terms of getting the kernel's IRE entries correct, it doesn't matter
so much where the physical wires go. It matters a whole lot what you do
with "ifconfig."
Ok, but when it is not connected it has no IP address (as it is
configured over DHCP) that can interfere with the multicast and the IP
setup. Maybe this is a problem when the address is static.
Or if you allow it to get the address first and then yank the cable. I
had no idea what your actual test case looked like, so I had to guess.
I looked at the time stamps
of the entries in the /var/adm/messages and they do not match the
freeze-ups by the minute.
I assume that refers to the NWAM messages previously reported. No, I
don't think those are the proximate cause of your problem.
I've been playing around with 'ifconfig<interface> unplumb' (as a
superuser of course) but it doesn't appear to do anything on e1000g2. As
a memory refresher, here's the setup:
e1000g1: 10.40.137.185, DHCP (the computer name is associated with this
address in the /etc/hosts file)
e1000g2: 10.40.137.171, DHCP (bridged network of the VM is attached to
this port)
rge0:<no IP address>, DHCP (no cable attached)
No error message comes after that command and when issuing 'ifconfig -a'
everything looks the same,
It's unclear what that means. My first guess would be that the command
was somehow misused, but I don't understand how. Perhaps you've
unplumbed IPv4 and were looking at IPv6 in the "ifconfig -a" output.
Or perhaps you're running NWAM, and that daemon is undoing your work
behind your back. You probably don't want to use NWAM with a reasonably
complex system configuration like this.
I think it is a bit strange that the changes only apply to the IPv4
settings but maybe it doesn't matter as the router only uses IPv4 (I
think). Hmm, I'm starting to wonder how netmasks and subnets work in
IPv6 as none appears to be specified in ifconfig -a.... I'm starting to
realize that you don't need nwam for dhcp.
and the '/sbin route monitor' doesn't yield
any messages.
That makes no sense. You should at least get a flurry of messages when
you unplumb a working interface.
Something's deeply amiss with your system, but I don't know what it
might be.
If I do the same on rge0 it will disappear in the IPv4
section of the 'ifconfig -a' output but remain in the IPv6 section. I
also see messages coming out of the route monitor; RTM_IFINFO,
RTM_DELETE and RTM_DELADDR... as a result of the unplumb command on rge0
which I think should be expected.
Yes; exactly.
I can see that e1000g2 is operating in Promiscuous state (Whatever that
means) which the other ethernet connections are not.
That's driven by applications. If you have an application (such as,
probably, VirtualBox) that sets the interface into "promiscuous mode"
(receive all unicast messages), then that's what you'll see.
Configuring IEEE 802 bridging will set the flag if that's used. (I
don't think you're using that here, though. It's configured using
"dladm create-bridge".)
Snoop/ethereal/wireshark will also put the interface into promiscuous
mode when capturing traffic.
IP itself will not.
"dladm" you say. I trust that VirtualBox does what it needs in that
regard and that I have to worry about it, for now.
I tried with 'ifconfig e1000g2 down' and the "UP" flag disappeared from
the port in 'ifconfig -a'.
This is probably confusing here, but those commands probably don't do
what you may think they do.
The "down" and "up" commands control the IP-layer IFF_UP flag. This is
an indication to IP itself and to higher-level applications that the IP
address itself is administratively available for use. It says nothing
about whether the interface is usable or connected, but rather that the
administrator wants applications to be able to use the address.
That's quite independent from plumbing -- which is the connection
between IP (or any network layer) and the datalink layer (Ethernet
driver). And, for that matter, it's also quite independent from the
IFF_RUNNING flag, which is set when the datalink layer tells the network
layer about status changes (such as a physical cable
connection/disconnection).
ifconfig(1M), unfortunately, comes from BSD which blurred the lines
between datalink and network layers. They're really quite distinct in
OpenIndiana (and Solaris before it), but the confused user interface in
ifconfig sadly leads users astray.
I see... I read on a forum that when unplumb fails it sometimes helps to
issue the 'ifconfig <IFP> down' command before unplumbing the
connection. But that was a BSD (don't know which flavor) forum.
Once again this only applies to IPv4, IPv6
remains unaffected. The route monitor yielded messages this time. The IP
address 10.40.137.171 is still there however and the bridged network
connection of the VM seems to be unaffected by this command (which would
be desired if that command really severed the IFP from the host).
It may well be going through the other interface in this case. I don't
think I have a good grasp on what you're actually trying to accomplish
here. It seems overly complex ...
If I try to unplumb it again (ifconfig e1000g2 unplumb) I get the error:
"ifconfig: cannot unplumb e1000g2: Invalid argument provided"
which is a bit strange.
Indeed.
The route monitor yields the message (in
digested form):
RTM_LOSING (Kernel Suspects partitioning)<DST,GATEWAY,NETMASK,IFA>
default 10.40.137.1 default default
right after this command.
Yikes! That's a good symptom that the IREs are broken due to the
misconfiguration. Nothing good can come of that message.
As a comparison; if I do the same on the rge0 which is already unplumbed
I get:
"ifconfig: cannot unplumb rge0: Interface does not exist"
and no messages on the route monitor.
Maybe I'm using a bad monkey wrench...
That seems likely. I think the whole thing was set in doubt when two
separate interfaces were configured up on the same subnet. Everything
past that point sounds like the experience of a test pilot. Some of the
control surfaces departed the craft in flight, but, hey, if that weren't
one of the possibilities, then we wouldn't need these brave souls.
I tried for a change to look into the Graphical Network Adminstration
tool in Gnome (/usr/lib/nwam-manager-properties) and the e1000g2
connection was disabled there even though 'ifconfig -a' showed that it
wasn't. I also verified with ping that this connection was not reponding
(ping 10.40.137.171) but that could be because I have done 'ifconfig ...
down' on it.
When I shut down the virtual machine, the e1000g2 connection disappeared
from 'ifconfig -a'. So it seems that the VM somehow blocked/prevented
ifconfig to apply the unplumbing and once the VM was shut down the
unplumbing kicked in first then.
I have now managed to disable the e1000g2 and the rge0 interface
permanently by the nwam-manager-properties program (by 'Edit"ing the
Network Profile). Both connections have now disappeared from 'ifconfig
-a' (also the IPv6 part is gone as well), even after a reboot, so I
consider them permanently unplumbed. It looks like this when issuing
'ifconfig -a':
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu
8232 index 1
inet 127.0.0.1 netmask ff000000
e1000g1: flags=1004843<UP,BROADCAST,RUNNING,MULTICAST,DHCP,IPv4> mtu
1500 index 2
inet 10.40.137.185 netmask ffffff00 broadcast 10.40.137.255
lo0: flags=2002000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv6,VIRTUAL> mtu
8252 index 1
inet6 ::1/128
e1000g1: flags=20002004841<UP,RUNNING,MULTICAST,DHCP,IPv6> mtu 1500 index 2
inet6 fe80::6a05:caff:fe01:da8e/10
The VirtualBox Qt GUI still recognizes all physical interfaces (e1000g1,
e1000g2 and rge0) even though they are disabled from the host which is good.
So I guess that once again the bets are on and all is good then...
Robin.
_______________________________________________
OpenIndiana-discuss mailing list
[email protected]
http://openindiana.org/mailman/listinfo/openindiana-discuss
