How to add proxy support to Jsoup?
You don't have to get the webpage data through Jsoup. Here's my solution, it may not be the best though.
URL url = new URL("http://www.example.com/");
Proxy proxy = new Proxy(Proxy.Type.HTTP, new InetSocketAddress("127.0.0.1", 8080)); // or whatever your proxy is
HttpURLConnection uc = (HttpURLConnection)url.openConnection(proxy);
uc.connect();
String line = null;
StringBuffer tmp = new StringBuffer();
BufferedReader in = new BufferedReader(new InputStreamReader(uc.getInputStream()));
while ((line = in.readLine()) != null) {
tmp.append(line);
}
Document doc = Jsoup.parse(String.valueOf(tmp));
And there it is. This gets the source of the html page through a proxy and then parses it with Jsoup.
You can easily set proxy
System.setProperty("http.proxyHost", "192.168.5.1");
System.setProperty("http.proxyPort", "1080");
Document doc = Jsoup.connect("www.google.com").get();
Jsoup 1.9.1 and above: (recommended approach)
// Fetch url with proxy
Document doc = Jsoup //
.connect("http://www.example.com/") //
.proxy("127.0.0.1", 8080) // sets a HTTP proxy
.userAgent("Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2") //
.header("Content-Language", "en-US") //
.get();
You may use also the overload Jsoup#proxy which takes a Proxy class (see below).
Before Jsoup 1.9.1: (verbose approach)
// Setup proxy
Proxy proxy = new Proxy( //
Proxy.Type.HTTP, //
InetSocketAddress.createUnresolved("127.0.0.1", 8080) //
);
// Fetch url with proxy
Document doc = Jsoup //
.connect("http://www.example.com/") //
.proxy(proxy) //
.userAgent("Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9.2.2) Gecko/20100316 Firefox/3.6.2") //
.header("Content-Language", "en-US") //
.get();
References:
- Connection#proxy(String,int)
- Connection#proxy(Proxy)
- Proxy class