We use palo-alto firewall as an internet gateway. We have 16 static ip-addresses. One is used for outbound traffic (users browse internet) . The rest is used for inbound traffic (mail server, webservers, etc). For redundant purposes we subscribe to second ISP. We buy 16 new static ip-addresses from new ISP. And here comes the hell with configuration. I've been reading for two days about BGP, PI addresses, AS numbers and other stuff. But I don't understand anything. Theory without practice and overall understanding is nothing. I call to these ISP's, and both providers say they won't configure any routes and won't sell AS numbers, try solve it by your selves. In our small asian country there is no LISP or any other cloud base routing solution. I don't know what to do next. Should I request AS number directly from APNIC? With policy based rules I may only configure outbound traffic redundancy. Is there any reliable solution to make redundant our small hosting? May it is possible to configure BGP without AS numbers and PI addresses?
-
Just a suggestion (maybe not practical): put your infrastructure in a data center / co-location facility. These typically have the redundancy you are looking for. Then as far as getting to those resources (which now have single IP addresses) you can use separate ISPs and internal routing protocols / VPNs to achieve site-to-dc redundancy. ...just a thought. – Ronnie Royston Jun 12 '15 at 15:55
3 Answers
Even if you could still get PI IPv4 addresses in Asia: if your ISPs don't want to route your IP addresses then there is nothing you can do. Tunnels and LISP could solve some of your problems (I use LISP here), but you already stated that this is not available in your region.
BGP is the protocol that is used to route your IP addresses from an AS. You need both to run BGP. Blocks of 16 addresses are too small to be routed with BGP anyway. Technically you could, but nobody will accept your routes.
If you want to have your own IP addresses and route them etc. you'll have to make some investments. Because APNIC ran out of IPv4 addresses for normal distribution you'll have to comply with some very strict rules. If I recall correctly the current rules are that you have to be multihomed already, must be able to justify 25% of the addresses (which would be 25% of 256 = 64) immediately and 50% (=128) within a year. Based on your current numbers that seems unlikely. If you could then you'd need to get an AS number from APNIC and you'll have to find ISPs that want to set up BGP sessions with you. This will probably be more expensive than your current contracts. And on top of that you'd have to study a lot to learn how internet routing and BGP works or you'll have to hire someone else to manage it for you. In addition to buying the equipment needed to do all of this.
In short: it's probably not worth it for your case.

- 6,670
- 22
- 33
-
1.Besides BGP there is no redundant solution to put a few servers for external usage, am I right? 2.What if we use two ext ip addresses for one internal server. And just point public DNS cname record to those two ip addresses? 3.What if we get those AS numbers and providers' routes, would it be enough to use just palo alto firewall? Or should we buy routers to use them as a gateways, one per ISP? – Алдар Apr 20 '15 at 20:27
-
1: BGP is *the* protocol to route addresses on the internet. 2: putting multiple addresses in DNS will make you dependent on all of them, *reducing* reliability. 3: using a single firewall would make that device a SPOF. A single ISP is probably more redundant than that, so you'll only make it worse... – Sander Steffann Apr 20 '15 at 20:31
-
You can see how I handled some level of redundancy with a single ISP over diverse circuits at http://networkengineering.stackexchange.com/questions/1745/inbound-bgp-load-balancing-from-same-isp-router. – generalnetworkerror Apr 21 '15 at 01:28
-
"2: putting multiple addresses in DNS will make you dependent on all of them, reducing reliability." Are you sure about this? Most applications try the first entry in the provided list then after a timeout period with no response move to the second entry. – cpt_fink Apr 22 '15 at 04:04
-
-
-
It depends on the OS and the browser and the type of timeout (connecting vs re-establishing a broken connection) and whether ICMP errors are properly propagated and handled. So it's difficult to give a simple answer. 300 seconds is normal. And this applies to every single connection, and loading a single web page usually uses multiple connections... – Sander Steffann Apr 28 '15 at 16:39
You can configure a Palo Alto Networks firewall to fail over to the other ISP. You need to set up two sets of NATs -- one for one ISP and one for the other -- or set two DMZs, one for one ISP and one for the other (or overlay two subnets on one interface). It will use both for inbound and will fail over to the second for outbound when one fails.
You can start reading here.

- 623
- 5
- 12
-
This won't work for inbound traffic for the servers. If one link goes down, how will external clients know how to reach the servers when their external IP address(es) change? DNS records with a low TTL may help for longish outages, but that is hugely unreliable and far from efficient. – stevieb Jun 12 '15 at 13:49
-
It absolutely DOES work. What you do is put an address from both ISPs on each service. You have two dns servers, dns01, dns02. Each has service IPs for one ISP's address space for services. Both are listed as dns servers for your domain. When everything is up, dns queries come to both and both addresses can be used. When one breaks the working one is used. – GeorgeB Jun 12 '15 at 15:07
-
Example: DNS01.foo.com has a zone file that lists ISP1 ip addresses for services. DNS02.foo.com has a zone file that lists ISP2 ip addresses for services. Both DNS servers are listed for the domain. When ISP1 goes down, DNS01 can no longer be reached. DNS02 serves ISP2 addresses to queries. Yes, it's "janky" but failover for static routed nets is janky. – GeorgeB Jun 12 '15 at 15:09
-
I see what you're saying. However, as the rest of the world already has cached records pointing to the old IPs, they won't look up the name again until their cache expires, at which time they will use the name of the up name server. Until then, their local pc cache contains an invalid record, as will their upstream DNS servers. You'd need to use a very low TTL, and not everyone honours those. So yeah, "janky", but I suppose it is better than nothing :) – stevieb Jun 12 '15 at 15:14
-
There is another way to do it in a vendor-specific way. In the PAN firewall enable DNS proxy. In the rule set for the interface to ISP1 you have static entries for ISP1 addresses for services. In the rule set for the interface to ISP2, entries for ISP2 addresses. Clients get address according to which interface the DNS query arrived on. – GeorgeB Jun 12 '15 at 15:17
-
Yes, that is why you put a short TTL on any records that are subject to failing over. Say 5 minutes. – GeorgeB Jun 12 '15 at 15:18
-
Better/easier hybrid approach. DNS server carries only ISP1 addresses. PAN DNS proxy enabled on only ISP2 interface. Rule set has ISP2 static IP addresses configured. Queries arriving on ISP1 go to DNS server. Queries to ISP2 get static ISP2 proxy rule entry. Only one zone file to maintain. Again, failing over static routed nets is tricky. – GeorgeB Jun 12 '15 at 15:26
-
I agree with you. No matter which approach is taken, there is guaranteed to be at minimum a decent portion of the userbase who will be in the dark for at least some time. So is the nature of the beast when one is not large enough to play in the BGP world – stevieb Jun 12 '15 at 15:30
-
At least you can list your smtp server with separate IP addresses and matching dns entries for each ISP. Add MX records for both and you have one less service to worry about. Depending on your requirements, buying a traffic manager (intelligent load balancer) service from a nearby IaaS provider might give you some of what you need. That way you always direct clients to the fixed IP addresses of the load balancers/traffic managers, who will determine the availability of your different addresses and services, and finally act as a proxy between the client and your service. – Roy Jul 07 '15 at 23:58
There is some way to load balancing without AS and PI.
For outbound it achieved by policy routing
For failover inbound traffic, is good to use dynamic DNS. When primary ISP changed, DNS name (with short enough TTL) of site changed to new IP and clients keep access to site.
Setting DNS to two IP simultaneous can make round robin IP selection on clients.
Periodical changing (with period near to DNS record TTL) between two IP also can make balancing. Same effect is using DNS server that support giving different IP to different clients.

- 754
- 6
- 15