Solved Bhyve - change ip "vm switch "

GlitchyDot

GlitchyDot

I did a mistake, by accident gave same ip address to BHYVE and my VM.
man vm switch - doe not show how to change ip. /vm/.config/system.conf if i edit IP - it does not work and cant connect trough VNC.
I forgot to add ssh to my VM so cant login and getting port22 Connection refused.
I have this IP set - 192.168.1.100/24 for BHYVE to get VNC and gave same ip to my VM inside BHYVE.
Do i have to use vm switch destroy and re-create Network ?

EDIT: My FreeBSD is headless WS
 
SirDice said:
vm console <name of vm> will get you a serial console connection to the VM.
Click to expand...
Tested once i could not get connected trough VNC. Maybe because OS was installed. If i boot new VM to install it ( iso ) - i can get console , but once installed - nope.
 
You need to ssh into the server then you'll be able to use vm console vm_name, VNC is for connecting to the VM, it's a different thing.
 
You probably need to restart /usr/local/etc/rc.d/vm, for the changes to system.conf be picked up. But this will likely also shutdown and restart all your VMs.

GlitchyDot said:
man vm switch - doe not show how to change ip.
Click to expand...
Oh?
Code: 
     switch address name a.b.c.d/xx|none
             Configure an IP address for the specified virtual switch. The
             address should be specified in CIDR notation. To remove an
             address, specify none in place of the address.

             If NAT funtionality is required, please configure an address on
             the switch to become the gateway address for guests. Source NAT
             rules can then be created using your choice of firewall or NAT
             daemon. If DHCP is desired, we recommend using a manual switch
             and configuring this by hand.
vm(8)

GlitchyDot said:
by accident gave same ip address to BHYVE and my VM.
Click to expand...
I'm a little confused here though, bhyve(8) doesn't have an IP address. Do you mean the host and VM have the same IP (thus creating an IP conflict)?

GlitchyDot said:
Tested once i could not get connected trough VNC.
Click to expand...
I think you are confusing the serial console and the "graphics-mode" VNC console.

Code: 
root@hosaka:~ # vm list
NAME            DATASTORE  LOADER     CPU  MEMORY  VNC           AUTO      STATE
case            default    bhyveload  4    4096M   -             Yes [4]   Running (2873)
jenkins         default    bhyveload  4    4096M   -             Yes [7]   Running (3691)
kdc             default    uefi       2    2048M   0.0.0.0:5900  Yes [1]   Running (1886)
lady3jane       default    bhyveload  4    16384M  -             Yes [3]   Running (2495)
centos9         stor10k    uefi       4    4096M   0.0.0.0:5900  No        Locked (hosaka.dicelan.home)
debian          stor10k    uefi       4    4096M   -             No        Stopped
errol           stor10k    bhyveload  2    4096M   -             Yes [9]   Running (4385)
fbsd-test       stor10k    bhyveload  2    4096M   -             Yes [13]  Running (5218)
fbsd-test-old   stor10k    bhyveload  4    4096M   -             No        Running (3921)
foreman         stor10k    uefi       4    8192M   -             No        Stopped
gl-runner-1     stor10k    bhyveload  4    4096M   -             Yes [12]  Running (5196)
gl-runner-2     stor10k    bhyveload  4    4096M   -             No        Stopped
gvm             stor10k    bhyveload  4    4096M   -             No        Stopped
haos            stor10k    uefi       4    4096M   -             No        Stopped
kibana          stor10k    bhyveload  4    8192M   -             Yes [6]   Running (3671)
phabricator     stor10k    bhyveload  1    4096M   -             No        Stopped
plex            stor10k    bhyveload  4    4096M   -             No        Stopped
riviera         stor10k    bhyveload  2    4096M   -             Yes [11]  Running (63543)
sdgame01        stor10k    uefi       4    8192M   0.0.0.0:5901  Yes [8]   Running (3904)
tessierashpool  stor10k    bhyveload  2    4096M   -             Yes [5]   Running (3403)
ubuntu          stor10k    uefi       2    4096M   -             No        Stopped
wintermute      stor10k    bhyveload  4    4096M   -             Yes [10]  Running (4696)
Only three of my VMs have a "graphics" (VNC) console (one of them is turned off), everything else has a serial console I can access through vm console ...

Code: 
     console name [com1|com2]
             Connect to the console of the named virtual machine.  Without
             network access, this is the primary way of connecting to the
             guest once it is running.

             By default this will connect to the first com port specified in
             the client configuration, which is usually com1.  Alternatively
             you can specify the com port to connect to.

             This looks for the nmdm(4) device associated with the virtual
             machine, and connects to it with cu(1).  Use ~+Ctrl-D to exit the
             console and return to the host.
 
SirDice said:
Oh?
Code: 
     switch address name a.b.c.d/xx|none
             Configure an IP address for the specified virtual switch. The
             address should be specified in CIDR notation. To remove an
             address, specify none in place of the address.

             If NAT funtionality is required, please configure an address on
             the switch to become the gateway address for guests. Source NAT
             rules can then be created using your choice of firewall or NAT
             daemon. If DHCP is desired, we recommend using a manual switch
             and configuring this by hand.
vm(8)
Click to expand...
Code: 
vm switch list
NAME    TYPE      IFACE      ADDRESS           PRIVATE  MTU  VLAN  PORTS
public  standard  vm-public  192.168.1.100/24  no       -    -     igb0
My arch linux vm has same ip address 192.168.1.100/24

vm(8) - so same command to set it and edit IP ?
in my case would be vm switch create -a 192.168.1.20/24 public ?
SirDice said:
You probably need to restart /usr/local/etc/rc.d/vm, for the changes to system.conf be picked up. But this will likely also shutdown and restart all your VMs.
Click to expand...
you mean : service restart /usr/local/etc/rc.d/vm ? I dont mind for all VM`s to get restarted as its only 1 :)



SirDice said:
I'm a little confused here though, bhyve(8) doesn't have an IP address. Don't you mean the host and VM have the same IP (thus creating an IP conflict)?
Click to expand...

Sorry, i think it should be VNC not bhyve or i completely miss-understand it ?
This is if i do from my HOST,
Code: 
vm list
NAME       DATASTORE  LOADER  CPU  MEMORY  VNC           AUTO  STATE
archlinux  default    uefi    12   6144M   0.0.0.0:5900  No    Running (26692)
If i do REMOTE i use
Code: 
vm switch list
NAME    TYPE      IFACE      ADDRESS           PRIVATE  MTU  VLAN  PORTS
public  standard  vm-public  192.168.1.100/24  no       -    -     igb0
 
Change was success but still cant connect with VNC to my VM and console does not work.
Is there some sort of cache for VNC ?
 
What does vm list show? Did you actually enable graphics="yes" on that VM? There won't be a VNC console if graphics isn't enabled.
 
SirDice said:
What does vm list show? Did you actually enable graphics="yes" on that VM? There won't be a VNC console if graphics isn't enabled.
Click to expand...
Yes, graphics was enabled, OS was installed and once i rebooted i got disconnect - suspecting due to IP clash.
Code: 
vm list
NAME       DATASTORE  LOADER  CPU  MEMORY  VNC           AUTO  STATE
archlinux  default    uefi    12   6144M   0.0.0.0:5900  No    Running (6196)
VNC error:
Code: 
Failed to connect to "192.168.1.20::5900":
Unable to connect to socket: Connection refused (61)
Attempt to reconnect?
 
What's the IP address of the host itself? What does ifconfig output?
 
Code: 
ifconfig
igb0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=a520b9<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,HWSTATS>
        ether 2c:fd:a1:34:33:8f
        inet 192.168.1.10 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::2efd:a1ff:fe34:338f%igb0 prefixlen 64 scopeid 0x1
        inet6 2a00:f502:150:1f89:2efd:a1ff:fe34:338f prefixlen 64 autoconf
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=23<PERFORMNUD,ACCEPT_RTADV,AUTO_LINKLOCAL>
igb1: flags=1008802<BROADCAST,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4e527bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,WOL_MAGIC,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6,HWSTATS,MEXTPG>
        ether 2c:fd:a1:34:33:90
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
lo0: flags=1008049<UP,LOOPBACK,RUNNING,MULTICAST,LOWER_UP> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet 127.0.0.1 netmask 0xff000000
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x3
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
vm-public: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=0
        ether 22:55:f6:1f:f9:ef
        inet 192.168.1.20 netmask 0xffffff00 broadcast 192.168.1.255
        id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
        maxage 20 holdcnt 6 proto rstp maxaddr 2000 timeout 1200
        root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
        member: tap0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 5 priority 128 path cost 2000000
        member: igb0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
                ifmaxaddr 0 port 1 priority 128 path cost 20000
        groups: bridge vm-switch viid-4c918@
        nd6 options=9<PERFORMNUD,IFDISABLED>
tap0: flags=1008943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        description: vmnet/archlinux/0/public
        options=80000<LINKSTATE>
        ether 58:9c:fc:10:ff:fb
        groups: tap vm-port
        media: Ethernet 1000baseT <full-duplex>
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        Opened by PID 6196
 
GlitchyDot said:
Why i dont need IP from the switch ?
Click to expand...
Why do you think it needs one? In the real world you might want to give a switch an IP address, but this is only to be able to manage that switch remotely. A switch (bridge(4) in this case) is a layer 2 connection, management is done from the host (although there isn't much to 'manage').
 
SirDice said:
Why do you think it needs one? In the real world you might want to give a switch an IP address, but this is only to be able to manage that switch remotely. A switch (bridge(4) in this case) is a layer 2 connection, management is done from the host (although there isn't much to 'manage').
Click to expand...
Oh , so basically if its local - you dont need switch for it , but if ... lets say outside connection - then you want to have one right ? Im really struggling with networking side - nothing makes sense to me when it comes to networking.
 
You might need one because of the way a bridge(4) hooks into the stack on the interfaces. That would only be problematic if you have a single interface for management and as 'uplink' for your VMs. But in that case you'd basically remove the IP address from the interface and move it to the switch.

I see you have two interfaces, which is nice. I would suggest using igb0 as your 'management' interface for the host, and connect your public switch to igb1, that will make things a lot easier. It does mean you will have to hook up both igb0 and igb1 to a physical (i.e. real) switch though.

And don't forget to up the interface, or else it'll be down and nothing will pass in or out.

Code: 
ifconfig_igb1="up"

No need for an IP address here, your VMs will be directly connected to the same layer 2 network because of the way a bridge(4) works.

I have a slightly more complicated construction as I've 'bundled' two interfaces to be used as 'uplinks' for my VMs:
Code: 
cloned_interfaces="lagg0"
ifconfig_igb0="inet 192.168.10.180 netmask 255.255.255.0 mtu 9000"
ifconfig_igb1="up mtu 9000 -rxcsum -rxcsum6 -txcsum -txcsum6 -lro -tso -vlanhwtso"
ifconfig_igb2="up mtu 9000 -rxcsum -rxcsum6 -txcsum -txcsum6 -lro -tso -vlanhwtso"
ifconfig_lagg0="laggproto lacp laggport igb1 laggport igb2"
And I have two switches, each tied to a different VLAN, but they both use the lagg0 interface as 'uplink' to the rest of my network:
Code: 
root@hosaka:~ # vm switch list
NAME     TYPE      IFACE       ADDRESS  PRIVATE  MTU   VLAN  PORTS
servers  standard  vm-servers  -        no       9000  11    lagg0
public   standard  vm-public   -        no       9000  10    lagg0
 
You must log in or register to reply here.
Back
Top