Routing, DNS resolution with Bond0

Question

I am having issues on 18.04 Server when I switch from dhcp to static IP on my device. The reason I was switching is that I was getting a new IP address on my server randomly even though my router had a reserved static IP address for my primary interface (eth0). I switched to static and now I'm having issues resolving /routing to my gateway. My setup has three interfaces (eth0 (primary), eth1, eth3). Don't know what is really causing the issues with the bond0 implementation. Data is moving ok within the network but outside is not and therefore I am unable to update the system unless I swap back to dhcp and live with the randomly changing dhcp address.

cat /etc/network/interfaces
# ifupdown has been replaced by netplan(5) on this system.  See
# /etc/netplan for current configuration.
# To re-enable ifupdown on this system, you can run:
#    sudo apt install ifupdown
#
# loopback
auto lo
iface lo inet loopback

# primary network card
auto eth0
iface eth0 inet manual
bond-master bond0
bond-primary eth0

# second network interface card
auto eth1
iface eth1 inet manual
bond-master bond0

# third network interface card
auto eth3
iface eth3 inet manual
bond-master bond0

# bond0 network interface
auto bond0
iface bond0 inet static
        address 192.168.0.100
        gateway 192.168.0.1
        netmask 255.255.255.0
        dns-nameservers 192.168.0.1 208.67.222.222 208.67.220.220
                bond_mode 6
                bond-miimon 100
                bond-slaves none

ifconfig
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST>  mtu 1500
        inet 192.168.0.100  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::6ef0:49ff:fe5e:c194  prefixlen 64  scopeid 0x20<link>
        ether 6c:f0:49:5e:c1:94  txqueuelen 1000  (Ethernet)
        RX packets 298900  bytes 19550931 (19.5 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2089172  bytes 3153193403 (3.1 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 6c:f0:49:5e:c1:94  txqueuelen 1000  (Ethernet)
        RX packets 354297  bytes 21749288 (21.7 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 15785  bytes 15597970 (15.5 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth1: flags=6147<UP,BROADCAST,SLAVE,MULTICAST>  mtu 1500
        ether 6c:f0:49:5e:c9:95  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth3: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST>  mtu 1500
        ether 00:14:5e:77:45:c6  txqueuelen 1000  (Ethernet)
        RX packets 296791  bytes 19306412 (19.3 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4487992  bytes 6786908424 (6.7 GB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 16  memory 0xfdee0000-fdf00000

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 7759  bytes 1630081 (1.6 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 7759  bytes 1630081 (1.6 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         192.168.0.1     0.0.0.0         UG    0      0        0 bond0
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 bond0

root@mediasvr:/home/xxxx# ping www.cnn.com
ping: www.cnn.com: Temporary failure in name resolution

root@mediasvr:/home/xxxx# nslookup www.cnn.com
;; connection timed out; no servers could be reached

root@mediasvr:/home/xxxx# ping 192.168.0.100
PING 192.168.0.100 (192.168.0.100) 56(84) bytes of data.
64 bytes from 192.168.0.100: icmp_seq=1 ttl=64 time=0.059 ms
64 bytes from 192.168.0.100: icmp_seq=2 ttl=64 time=0.053 ms

--- 192.168.0.100 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1028ms
rtt min/avg/max/mdev = 0.053/0.056/0.059/0.003 ms

**root@mediasvr:/home/xxxx# ping 192.168.0.1**
PING 192.168.0.1 (192.168.0.1) 56(84) bytes of data.
From 192.168.0.100 icmp_seq=4 Destination Host Unreachable
From 192.168.0.100 icmp_seq=5 Destination Host Unreachable
From 192.168.0.100 icmp_seq=6 Destination Host Unreachable
From 192.168.0.100 icmp_seq=16 Destination Host Unreachable
From 192.168.0.100 icmp_seq=17 Destination Host Unreachable
From 192.168.0.100 icmp_seq=18 Destination Host Unreachable
From 192.168.0.100 icmp_seq=19 Destination Host Unreachable
From 192.168.0.100 icmp_seq=20 Destination Host Unreachable

--- 192.168.0.1 ping statistics ---
23 packets transmitted, 0 received, +8 errors, 100% packet loss, time 22504ms

root@mediasvr:/home/xxxx# systemd-resolve --status
Global
         DNS Servers: 192.168.0.1
                      208.67.222.222
                      208.67.220.220
          DNSSEC NTA: 10.in-addr.arpa
                      16.172.in-addr.arpa
                      168.192.in-addr.arpa
                      17.172.in-addr.arpa
                      18.172.in-addr.arpa
                      19.172.in-addr.arpa
                      20.172.in-addr.arpa
                      21.172.in-addr.arpa
                      22.172.in-addr.arpa
                      23.172.in-addr.arpa
                      24.172.in-addr.arpa
                      25.172.in-addr.arpa
                      26.172.in-addr.arpa
                      27.172.in-addr.arpa
                      28.172.in-addr.arpa
                      29.172.in-addr.arpa
                      30.172.in-addr.arpa
                      31.172.in-addr.arpa
                      corp
                      d.f.ip6.arpa
                      home
                      internal
                      intranet
                      lan
                      local
                      private
                      test

Link 8 (bond0)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 6 (eth4)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 5 (eth3)
      Current Scopes: none
       LLMNR setting: yes
MulticastDNS setting: no
      DNSSEC setting: no
    DNSSEC supported: no

Link 4 (eth2)
      Current Scopes: none
       LLMNR setting: yes

I am including the additional information requested.

root@mediasvr:/etc/netplan# cat 50-cloud-init.yaml

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
network:
    version: 2
    renderer: networkd
    ethernets:
      bond0:
        dhcp4: no
        dhcp6: no
        addresses: [192.168.0.100/24]
        gateway4: 192.168.0.1
        nameservers:
            addresses: [208.67.222.222, 208.67.220.220]

root@mediasvr:/etc/netplan# ls -al /etc/resolv.conf

lrwxrwxrwx 1 root root 27 Apr 12 14:40 /etc/resolv.conf -> /run/resolvconf/resolv.conf

root@mediasvr:/etc/netplan# cat /etc/resolv.conf

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
# 127.0.0.53 is the systemd-resolved stub resolver.
# run "systemd-resolve --status" to see details about the actual nameservers.

nameserver 208.67.222.222,
nameserver 208.67.220.220
nameserver 127.0.0.53

@heynnema - Update

So I changed both the interface and the .yaml file and applied the netplan without errors (debug) however, I am still having network issues:

1. I can ping my interface
2. Team bond is fine according to /proc/net/bonding/bond0 (static) However,
3. Route command delays to pop up
4. Unable to ping the gateway
5. Unable to resolve DNS

Here are the updates I performed:

cat /etc/network/interfaces

# ifupdown has been replaced by netplan(5) on this system.  See
# /etc/netplan for current configuration.
# To re-enable ifupdown on this system, you can run:
#    sudo apt install ifupdown
#
# loopback
auto lo
iface lo inet loopback

/etc/netplan/50-cloud-init.yaml

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
network:
  version: 2
    renderer: networkd
    ethernets:
      eth0:
        dhcp4: no
      eth1:
        dhcp4: no
      eth3:
        dhcp4: no
    bonds:
      bond0:
        interfaces: [eth0, eth1, eth3]
        addresses: [192.168.0.100/24]
        gateway4: 192.168.0.1
        nameservers:
          search: [local]
          addresses: [208.67.222.222, 208.67.220.220]
        parameters:
          mode: balance-alb
          mii-monitor-interval: 1
          gratuitious-arp: 5

@heynnema - Update (Thank you for your expertise/time. I believe we are getting closer to resolving this problem)

I enabled an extra unused interface and set it to dhcp in the yaml file and applied via netplan. I kept the team bond the same on the existing 3 nics (eth1-eth3)

eth4: dhcp4: true optional: true

root@mediasvr:/home/raul# route

Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         _gateway        0.0.0.0         UG    0      0        0 bond0
default         _gateway        0.0.0.0         UG    100    0        0 eth4
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 eth4
192.168.0.0     0.0.0.0         255.255.255.0   U     0      0        0 bond0
_gateway        0.0.0.0         255.255.255.255 UH    100    0        0 eth4

root@mediasvr:/home/raul# netstat -rn

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG        0 0          0 bond0
0.0.0.0         192.168.0.1     0.0.0.0         UG        0 0          0 eth4
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 eth4
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 bond0
192.168.0.1     0.0.0.0         255.255.255.255 UH        0 0          0 eth4

root@mediasvr:/home/raul# traceroute 192.168.01

traceroute to 192.168.01 (192.168.0.1), 64 hops max
  1   192.168.0.1  0.569ms  0.359ms  0.370ms

1. I am able to ping/perform traceroute to the gateway which I couldn't before.
2. I can perform DNS lookups
3. I can ssh into the server remotely via dhcp address

However, 4. I am unable to connect to repositories to get updates - connect (113: No route to host) Could not connect to archive.ubuntu.com:80 (91.189.88.149). 5. I am unable to connect to the plex media server which is now listening on the dhcp address

tcp        0      0 mediasvr:32400          192.168.0.11:10558      TIME_WAIT
tcp        0      0 mediasvr:49206          mediasvr:32400          ESTABLISHED

@heynnema - Updated yaml file (would you rather continue this in a chat?)

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
network:
  version: 2
    renderer: networkd
    ethernets:
      eth0:
        dhcp4: no
        optional: true
      eth1:
        dhcp4: no
        optional: true
      eth3:
        dhcp4: no
        optional: true
      eth4:
        dhcp4: true
        optional: true  
    bonds:
      bond0:
        interfaces: [eth0, eth1, eth3]
        addresses: [192.168.0.100/24]
        gateway4: 192.168.0.1
        nameservers:
          search: [local]
          addresses: [192.168.0.1, 208.67.222.222, 208.67.220.220]
        parameters:
          mode: balance-alb
          mii-monitor-interval: 1
          gratuitious-arp: 5

@heynnema - Update - I've applied your suggestion on the yaml changes. It appears that it has improved a couple of things. Specifically;

1. I can now ping both eth4 and bond0 internally/externally and the gateway.
2. DNS resolution works
3. I can SSH into the server

However, 1. Plex uses the first nic it sees which in this case is the dhcp interface 2. I am unable to interact with the internet

UPDATED Yaml file
    network:
      version: 2
      renderer: networkd
      ethernets:
        eth0:
         dhcp4: no
         optional: true
        eth1:
         dhcp4: no
         optional: true
        eth3:
         dhcp4: no
         optional: true
        eth4:
         dhcp4: true
         optional: true
      bonds:
        bond0:
         interfaces: [eth0, eth1, eth3]
         addresses:
           - 192.168.0.100/24
         #gateway4: 192.168.0.1
         routes:
           - to: 0.0.0.0/0
             via: 192.168.0.1
             metric: 100
         nameservers:
           search: [local]
           addresses: [208.67.222.222, 208.67.220.220]
         parameters:
           mode: balance-alb
           mii-monitor-interval: 1
           gratuitious-arp: 5

root@mediasvr:/home/raul# netstat -rn
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         192.168.0.1     0.0.0.0         UG        0 0          0 bond0
0.0.0.0         192.168.0.1     0.0.0.0         UG        0 0          0 eth4
0.0.0.0         192.168.0.1     0.0.0.0         UG        0 0          0 bond0
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 eth4
192.168.0.0     0.0.0.0         255.255.255.0   U         0 0          0 bond0
192.168.0.1     0.0.0.0         255.255.255.255 UH        0 0          0 eth4

Answer 1

I have found that in Ubuntu Server 18.04, there is a new way to communicate w/ exterior connections. If you look in /etc/netplan/, there will be a file called 50-cloud-init.yaml.

This is the file, supposedly, that needs to be altered to work w/ static ip addresses. I actually posted a familiar question and there was this fellow that helped out. See here: RE: Getting Ubuntu Server to Show Web Page URL instead of Just my IP Address.

...

If this helps, this is a good notation to make. I also found some similar instances on this site that are dedicated to the server side of Ubuntu.

Seth

P.S. I do not have a 100% cure for this static ip issue yet but my page is up and running w/ "Port Forward" on my router as of now. I actually tried to change out my .yaml file, the file I described earlier in /etc/netplan/, w/out satisfaction. Look to netplan(5) for starter info. and a good set of conclusions on the daemon. Also, w/ this being a resort, you can get a third party like no-ip to host your static ip for you if this is too large of an issue.

...

So,

For Static IP Addresses: Use your favorite editor w/ the file /etc/netplan/50-cloud-init.yaml or use a third party like no-ip.com.

For setting up your DNS: Use bind9, e.g. sudo apt install bind9. You have to uncomment your file at /etc/bind/named.conf.options and set your IP:

   forwarders { 
  8.8.8.8; 
  8.8.4.4; 
};

I have been reading out of Lacroix's book, "Mastering Ubuntu Server." It is dedicated to the new release of Ubuntu Server during specific sections.

Anyway...the above code should not be commented and it, as is, is directed at google's DNS servers. You can point them to your ISP's DNS servers too.

Now...restart the bind service w/ sudo systemctl restart bind9.

Let me know if this is clear or too muddy. I will try to make sense out of your situation more as time persists. Please let me know if this supports you cause.

Oh! Are you using a router and access point for your server or just a router?

To make a gateway w/ an access point and router, use this command:

echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward

I got that command from the book, too.

Now, to make that command work after reboots, go to the file, /etc/sysctl.conf, to change things. Use this change.

#net.ipv4.ip_forward=1

to:

net.ipv4.ip_forward=1

Just uncomment out the hash mark.

This should stop the reboot from cancelling out your server w/ Ubuntu Server.

Answer 2

• I'm assuming that eth0 is your WAN connection

• I'm not bonding 3 interfaces in my .yaml

• let eth0 pick up any IP address via dhcp4 for right now

• we can set a router reserved address for eth0 later...

  • sudo lshw -C network or ifconfig or ip addr# identify proper MAC for eth0
  • set router reserved dhcp using that MAC, and the desired IP

• some tweaking will be required along the way

sudo netplan --debug generate # generate config files

sudo netplan apply # apply new configuration

reboot # and verify proper operation

---

network:
  version: 2
    renderer: networkd
    ethernets:
      eth0:
        dhcp4: true
        optional: true
      eth1:
        dhcp4: false
        optional: true
      eth3:
        dhcp4: false
        optional: true
      eth4:
        dhcp4: false
        optional: true
    bonds:
      bond0:
        interfaces: [eth1,eth3,eth4]
        addresses: [192.168.0.100/24]
        #gateway4: 192.168.0.1 # not used with routes
        routes:
          - to: 0.0.0.0/0
            via: 192.168.0.1
            metric: 100
        nameservers:
          search: [local]
          addresses: [208.67.222.222,208.67.220.220]
        parameters:
          mode: balance-alb
          mii-monitor-interval: 1
          gratuitious-arp: 5