Skip to main content

Posts

Showing posts from 2009

Screen Scraping using Java

1. Here we have to use the basic package com.gargoylesoftware.htmlunit; 2. How to get a page using com.gargoylesoftware.htmlunit.WebClient WebClient webClient = new WebClient(); webClient.setThrowExceptionOnFailingStatusCode(false); webClient.setThrowExceptionOnScriptError(false); webClient.setJavaScriptEnabled(false); webClient.setTimeout(60000); // 1 minute URL url = new URL("http://www.iberia.com/?language=en"); // Now you have the url into homepage HtmlPage homepage = (HtmlPage)webClient.getPage(url); 3. taking a form from a page // single form HtmlForm frm1 = (HtmlForm) homepage .getFormByName("airPaxFormSimpl"); // multiple form List listFormss = homepage.getForms(); HtmlForm frm1 = (HtmlForm)listFormss.get(5); 4. How to set or get value from a text box HtmlTextInput p0name = (HtmlTextInput) frm1.getInputByName("passengers[0].name"); p0n