Fixing my VPN redirect problem on Mac OSX
1. Introduction
I use a local domain for development on my Mac, and this allows me to navigate to dev.belongstonobody.com:3000 instead of localhost:3000 via a simple /etc/hosts configuration.
Whenever I turn on my VPN however, things no longer work. Instead of showing my rails app, I would get redirected to “rss2search.com” and “search.conduit.com” with a message saying that the site is blocked. What makes it even more odd is that the problem only occurs when using Google Chrome, as it works for Firefox, curl, or ping. In here, I detail my solution to this problem.
2. Analyzing HTTP
Focusing first on how we’re getting to “search.conduit.com”, I used Wireshark to analyze HTTP request before and after VPN is enabled.
Without VPN
With VPN
When VPN is off, navigating to dev.belongstonobody.com:3000 in Chrome would show the result below. We can see that our browser is making a GET request to port 3000 which is the expected behavior and the local server was able to respond to the request appropriately with a HTTP response code of 200.
Once we turn on the VPN, navigating to dev.belongstonobody.com:3000 in Chrome shows something different.
The first thing we see is that the browser is now talking to port 8555 instead of 3000. The second thing we notice is that our request is being redirected with a 302 response code. Looking at the Location response header, we see the infamous “rss2search.com” url with a query parameter pointing to our site. And with the chain of redirects, we are finally led to the “search.conduit.com” webpage. You may wonder about the 200 OK response served by rpt.anchorfree.net/afrpt.php, but all it contains is a javascript code that sets window.location.href to “search.conduit.com”, and hence the unbroken redirect chain.
Local Proxy
It turns out that the redirects are caused by a local proxy, as opposed to Chrome. To be specific, port 8555 is actually binded to af_proxy, a program belonging to my VPN client, which now acts as local proxy for all HTTP requests made by Google Chrome. Navigating to chrome://net-internals/#proxy in Google Chrome confirms that proxy settings has been changed to use port 8555 after VPN was turned on.
Now, we want to find out under what circumstances the local proxy triggers a redirect. To do that, I tried other examples such as non-existing domain that’s not listed on my /etc/hosts. Using http://www.asdflfjalsdkfjaslkdf, I see that the same redirect chain visible.
Based on the behavior above, it seems like redirect happens whenever my VPN client is unable to resolve a domain name. It’s unable to resolve dev.belongstonobody.com probably because it’s not using Mac’s DNS resolver, mDNSResponder, which reads /etc/hosts entries.
3. Analyzing DNS
When I analyzed the outgoing DNS requests of different urls through wireshark, I noticed something interesting. I can see the DNS queries when the url scheme is https. However, for non-https sites, I don’t see them at all.
HTTPS - With DNS
HTTP - No DNS
4. Cause of Redirect
What we’ve seen above strongly suggests that the DNS resolution for non https sites (including my local domain) is actually being done on the VPN server, where my /etc/hosts entry would simply be irrelevant as its unreachable. If Google Chrome does not use a proxy, it makes a DNS lookup first before making a TCP connection to the server it wants to talk to. However, with VPN turned ON and the requests being forwarded to a local proxy, Chrome no longer does a DNS lookup. Instead, it offloads the responsible to the proxy, which in turn decides that it should be done on the VPN server for non-https sites (including our local domain).
5. Solution: Bypassing the Proxy
[EDIT] Turns out a local proxy is actually not needed at all for a VPN setup to function correctly as suggested by a redditor. Instead of specifying a specific domain to bypass a proxy as shown below, one can specify ALL requests to bypass proxy by entering “*” in “Bypass proxy settings for these Hosts & Domains” setting.
To fix the redirect problem, I simply bypass the local proxy whenever the URL matches my local domain. It can be accessed by going to “System Preferences” -> “Network” -> “Proxies”. Then under “Bypass proxy settings for these Hosts & Domains”, specify the domain that will bypass the proxy - in my case, dev.belongstonobody.com. It is important that one restart their computer for changes to take effect (at least for me in Mountain Lion).