0

I am having ubuntu 18.04 on a HP Omen laptop.

I have noticed that whenever I visit a website that I haven't visited before or I havent visited for the last hours it takes too many seconds to load(2-5sec). After the first time if i visit later it takes 'normal' time(0.1-0.5sec).

What I have tried/to be mentioned:

  • I have tested it on both Chrome and Firefox with the same issue on both.
  • wireless and wired test: same issue occurs.
  • I have disabled hardware accelerator from chrome and didn't helped.
  • another computer running on the same network with windows OS doesn't have this issue.

And some tests(taking advice from heynnema):

time host www.booking.com

When running the above for first time run (2.4sec) 

real    0m2,410s
user    0m0,010s
 sys    0m0,000s

Running it again (0.01sec):

real    0m0,011s
user    0m0,000s
 sys    0m0,010s

Tested also on other websites where I saw some reaching even 6seconcs when running for first time.

Edit1

Also:

ls -al /etc/resolv.conf
lrwxrwxrwx 1 root root 39 Σεπ   3 21:00 /etc/resolv.conf -> ../run/systemd/resolve/stub-resolv.conf

and:

cat /etc/resolv.conf
# This file is managed by man:systemd-resolved(8). Do not edit.
#
# This is a dynamic resolv.conf file for connecting local clients to 
the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "systemd-resolve --status" to see details about the uplink DNS 
servers
# currently in use.
#
# Third party programs must not access this file directly, but only 
through the
# symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a 
different way,
# replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported 
modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0

and:

ip addr | grep mtu
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
3: wlo1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
4: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
5: br-44cc05437844: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
6: br-a928e5f9d267: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default

and /etc/systemd/resolved.conf:

# /etc/nsswitch.conf
#
# Example configuration of GNU Name Service Switch functionality.
# If you have the `glibc-doc-reference' and `info' packages installed, 
try:
# `info libc "Name Service Switch"' for information about this file.

passwd:         compat systemd
group:          compat systemd
shadow:         compat
gshadow:        files

hosts:          files mdns4_minimal [NOTFOUND=return] dns myhostname
networks:       files

protocols:      db files
services:       db files
ethers:         db files
rpc:            db files

netgroup:       nis

and /etc/systemd/resolved.conf: 

#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See resolved.conf(5) for details

[Resolve]
#DNS=
#FallbackDNS=
#Domains=
#LLMNR=no
#MulticastDNS=no
#DNSSEC=no
#Cache=yes
#DNSStubListener=yes

Edit2

Also using the answer from Use curl to find out which part of the process of loading a website is giving you trouble (used www.booking.com for the example below) I have the following result:

  DNS lookup                          :  6,516376
  Connect to server (TCP)             :  6,718620
  Connect to server (HTTP/S)          :  0,000000
  Time from start until transfer began:  6,718712
  Time for redirection (if any)       :  0,000000
  Total time before transfer started  :  6,950434

         Total time                   :  6,950476
         Size of download (bytes)     :  0
         Average d/l speed (bytes/s)  :  0,000

Seems like DNS lookup is taking the longest.

Any idea why is this happening? Thanks

Edit3

I have created a new resolv conf file resolv8.conf in /run/systemd/resolve with the following inside(google dns):

nameserver 8.8.8.8
nameserver 8.8.4.4

and a new symlink:

sudo ln -s /run/systemd/resolve/resolv8.conf /etc/resolv.conf

and now host -v .. and dns lookup timings are reasonable. However, there is 'collateral damage'. In every restart I loose the resolv8.conf file so I have to rewrite it and create new symlink again. Also with these changes chrome browsing works normally but not Firefox where I cant access any website(?? maybe i need to do something with networkManager?).

I tottaly dont get why is quicker now though.

Also is there a good way to make the changes permanent? Will this affect when I am connected to other rooters except of my house?

Edit4

Here a screenshot of my rooter configurations in case there are some wrong defined configs

enter image description here

Edit5

Testing on fresh 'try ubuntu 18.04' from usb:

Configurations seems the same with the above and also:

ip addr | grep mtu
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000  
2: eno1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN group default qlen 1000  
3: wlo1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000

and 

host -v www.airbnb.com | grep -i received
Received 190 bytes from 127.0.0.53#53 in 54 ms
Received 39 bytes from 127.0.0.53#53 in 432 ms
Received 39 bytes from 127.0.0.53#53 in 1847 ms
host -v www.booking.com | grep -i received
Received 69 bytes from 127.0.0.53#53 in 34 ms
Received 35 bytes from 127.0.0.53#53 in 823 ms
Received 35 bytes from 127.0.0.53#53 in 1039 ms

Seems that there is the same issue. Slow dns lookup

Mpizos Dimitris
  • 111

1 Answers1

2

Your problem is with the MTU setting for your DSL connection.

There's a MTU setting in Ubuntu's network configuration, and a WAN MTU setting in your router.

For DSL, a common MTU setting is 1492. Just go ahead and try this value first and see if your web sites are now accessible.

To determine the correct setting, start with all MTU settings = 1500 and VPN = off. (VPN requires different testing).

In terminal:

    ping [-c count] [-M do] [-s packet_size] [host]

The options used are:

  • c count: number of times to ping
  • M hint: Select Path MTU Discovery strategy. may be either do (prohibit fragmentation, even local one), want (do PMTU discovery, fragment locally when packet size is large), or dont (do not set DF flag).
  • s packet_size: Specifies the number of data bytes to be sent.

You should always start at 1472 and work your way down by 10 each time. Once you get a reply, go up by 1 until you get a fragmented packet. Take that value (last good value) and add 28 to the value to account for the various TCP/IP headers. Eg. let's say that 1452 was the proper packet size (where you first got an ICMP reply to your ping). The actual MTU size would be 1480, which is the optimum for the network we're working with.

    ping -c 4 -M do -s 1472 8.8.8.8 # this will probably show fragmentation

    ping -c 4 -M do -s 1462 8.8.8.8 # may show fragmentation

    ping -c 4 -M do -s 1452 8.8.8.8 # no fragmentation?

    ping -c 4 -M do -s 1453 8.8.8.8 # still no fragmentation?

reference: [How to determine the proper MTU size with ICMP pings][1]

  [1]: http://muzso.hu/2009/05/17/how-to-determine-the-proper-mtu-size-with-icmp-pings

Update #1:

We need to check if your DNS servers have a problem with the options edns0 in /etc/resolv.conf.

/etc/resolv.conf is a symlink that goes to one of three places. Do a:

ls -al /etc/resolv.conf # note the original symlink setting

sudo cat /etc/resolv.conf # note the current file contents

sudo rm -i /etc/resolv.conf # remove the symlink

sudo ln -s /run/resolvconf/resolv.conf /etc/resolv.conf # new symlink

sudo cat /etc/resolv.conf # note the current file contents

retest DNS using:

host -v www.booking.com | grep -i received # use a different web site every time

sudo rm -i /etc/resolv.conf # remove the symlink

Depending on the results, you can also try this symlink:

sudo ln -s /run/systemd/resolve/resolv.conf /etc/resolv.conf # a different symlink

sudo cat /etc/resolv.conf # note the current file contents

retest DNS using:

host -v www.booking.com | grep -i received # use a different web site every time

sudo rm -i /etc/resolv.conf # remove the symlink

If one symlink works better than the other, keep it for now. If none work better, set it back to original:

sudo ln -s /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf # reset to original symlink

heynnema
  • 70,711
  • Thanks heynnema for the answer and time!Followed the process to find correct mtu setting. I have set to 1492(1464+28) with ip link set dev wlo1 mtu 1492 and run again time host www.booking.com to check timing but unfortunately didn't resolve the issue. – Mpizos Dimitris Nov 30 '19 at 18:30
  • @MpizosDimitris check the setting in your NM GUI wl01 settings pane. Change it to 1492 and disconnect/reconnect to your wireless and retest. – heynnema Nov 30 '19 at 18:36
  • I coundnt find filed for mtu for wireless on the GUI, so I tried the above with wired connection where I replaced mtu from 'automatic' to 1492 in the GUI. But no luck. Still got high numbers (reached ~8sec for time host www.booking.com) – Mpizos Dimitris Nov 30 '19 at 18:57
  • Do you know how to ln -s and rm or mv in the terminal? – heynnema Nov 30 '19 at 19:04
  • yes I know the above commands – Mpizos Dimitris Nov 30 '19 at 19:20
  • We need to see if your DNS servers have a problem with the options edns0 line in /etc/resolv.conf. Give me a few minutes to write an update to my answer, and you can give it a try. – heynnema Nov 30 '19 at 19:24
  • @MpizosDimitris also show me cat /etc/nsswitch.conf. – heynnema Nov 30 '19 at 19:46
  • @MpizosDimitris also show me cat /etc/systemd/resolved.conf – heynnema Nov 30 '19 at 19:51
  • updated question. I tried for /run/systemd/resolve/resolv.conf with not any mentionable change. /run/resolvconf/resolv.conf doesnt exist. To apply them do I need to have mtu set to 1492? (btw gui and terminal mtu settings do not seem to be synchronized. How do I know which mtu is set for real?) – Mpizos Dimitris Nov 30 '19 at 20:01
  • @MpizosDimitris Ok, set the symlink back to original. Yes, with DSL, 1492 is correct, and your web sites/etc should load faster. I'd believe the NM GUI settings. Give me the two cat files that I need to look at please. Also, with MTU=1492 on en01, see if your perceive any faster web site load times... or sites that wouldn't load before, load now. – heynnema Nov 30 '19 at 20:25
  • I updated the question from before for what you asked. fyi all sites are loading, but the first time i go to one site it takes some time. – Mpizos Dimitris Nov 30 '19 at 20:32
  • @MpizosDimitris Thanks for the cat files. They look fine. Using host -v www.ebay.com | grep -i received, tell me what times you see, when run twice. Also show me ping 8.8.8.8. – heynnema Nov 30 '19 at 20:33
  • around ~210ms for first time, 0ms for second time. ping ~40-90ms – Mpizos Dimitris Nov 30 '19 at 20:35
  • Yeah, that's too long. However, I'm out of ideas. Sorry. – heynnema Nov 30 '19 at 20:36
  • Thanks for your time heynnema. I appriciate it:) – Mpizos Dimitris Nov 30 '19 at 20:36
  • @MpizosDimitris Please let me know if you ever find a solution. Thanks! – heynnema Nov 30 '19 at 20:38
  • sure. definition of solution should be that running twice host -v ... should give close values for run1 and run2 right? – Mpizos Dimitris Nov 30 '19 at 20:52
  • 1
    @MpizosDimitris no. When using a different web site every time... First time should be longer, maybe ~80ms. Second time should be less than 5ms. – heynnema Nov 30 '19 at 20:54
  • I have updated my question with a 'semi-work around'. It would be nice if you have any insight to share. – Mpizos Dimitris Dec 04 '19 at 09:50
  • 1
    ubuntu will most likely use the dns server from your router and then use them to feed the local ubuntu dns. if you visit a new site you don't have a cached ip record of that name. this is def. a dns problem. the servers your router points to is too slow. it takes them seconds to answer. you don't need to use google, there are other ntp pools available. this is much better, as there are always 4 or even more servers behind that dns pool name. if you can get into your router and choose the pool server for your region, try changing them there and see what happens. https://www.pool.ntp.org/zone/de – s1mmel Dec 04 '19 at 10:12
  • @s1mmel I do understand that there is cached ip lookup thats why second time loads quick. can u elaborate a bit more about what should I change on rooter and in which dns server my rooter should point? I have updated my question with info from my rooter. – Mpizos Dimitris Dec 04 '19 at 11:38
  • Try with the option AssignIspDNS, check the box, see if this feels different. Also, make sure to unset the changes you made to the resolv.conf. ATM your router is the DNS-server, means he will ask the dns entries with the isp, with the checkbox the router should set the isps dns servers, instead of your routers dns. i keep my fingers crossed, that this will speed up your dns queries. there might be a downside, it could be that the internal dns names won't work anymore. but you can easily fix that with entries in /etc/hosts – s1mmel Dec 04 '19 at 13:21
  • one more thing, there are empty fields for dns servers2 and 3. you can also type in additional dns servers from outside, server1 will be asked first, if he doesnt know, server2 is asked, then number3. this might be the better solution, because internal dns will still be working. try this solution first. – s1mmel Dec 04 '19 at 13:24
  • @s1mmel you mean maybe to add google dns in server2 and 3 as first try? or other endpoints? – Mpizos Dimitris Dec 04 '19 at 14:24
  • 1
    @MpizosDimitris no, your workaround won't work. There's no such actionable file named resolv8.conf in the system software. Set it all back as I describe in my answer. I don't think we tried this... Boot to a Ubuntu Live DVD/USB and try the host -v command again, and see if the times are more reasonable. – heynnema Dec 04 '19 at 16:11
  • @s1mmel dns and ntp are two different things. – heynnema Dec 04 '19 at 16:13
  • @heynnema I am gonna try to boot through ubuntu live usb tmr(I am afk). what do you mean by workaround will not work? Why with my 'solution' i experience quick host -v www... then? My undersanding on general is that I have slow dns lookup. Is that hypothesis correct? Just to know in which direction to search. thanks:) – Mpizos Dimitris Dec 04 '19 at 16:42
  • Yes, you have slow DNS lookups. Try the Ubuntu Live and host -v and report back. – heynnema Dec 04 '19 at 16:48
  • @heynema yeah you are right. i totally messed this one up. but my tip should still be valid. – s1mmel Dec 05 '19 at 07:45
  • @Mpizos Yes, that was my intention. Add the google dns, or others you know and put them in as server 2 or 3 and see how this works for you. if it does not. check the box with the ipsdns. when i see this correctly your request will then be forwarded directly to your isp dns. just dont use the link i gave you, i totally messed this one up. these are pools for time servers not name resolution. im terribly sorry for the mislead. – s1mmel Dec 05 '19 at 07:48
  • @MpizosDimitris status please... – heynnema Dec 11 '19 at 14:17
  • @heynnema my apologies for not updating. I haven't worked on it yet, too busy. I will this weekend and update. – Mpizos Dimitris Dec 11 '19 at 14:26
  • @heynnema I have updated my answer: same case with the a fresh 'try ubuntu' – Mpizos Dimitris Dec 22 '19 at 10:11
  • @MpizosDimitris OK, the problem isn't with your Ubuntu then, it's elsewhere. Lets tinker with some router settings. In the router screenshot you've loaded, change DNS Server 1/2/3 from 192.168.2.1/0/0 to 8.8.8.8/8.8.4.4/null, and gateway to null (you may have to reset the gateway to 192.168.2.1 if it breaks everything). Retry DNS... but remember, for the test to be valid, you have to select a different remote site every time. – heynnema Dec 22 '19 at 15:20
  • @heynnema I am using this laptop in 3 different locations(so 3 different rooters) with same behavior observed. So probably is not related to the specific rooter settings. I need to unpack my old laptop to use 'try ubuntu' to see whats the behavior there. I ll update. thanks for your time till now:) – Mpizos Dimitris Dec 23 '19 at 08:21
  • OK, please keep me posted. Note that they may move this discussion to chat, due to its length. – heynnema Dec 23 '19 at 15:31