HTML is core of web, all the page you see in internet are HTML, whether they are dynamically generated by JavaScript, JSP, PHP, ASP or any other web technology. Your browser actually parse HTML and render it for you. But what would you do, if you need to parse an HTML document and find some elements, tags, attributes or check if a particular element exists or not from Java program. If you have been in Java programming for some years, I am sure you have done some XML parsing work using parsers like DOM and SAX, but there is also good chance that you have not done any HTML parsing work. Ironically, there are few instances when you need to parse HTML document from core Java application, which doesn't include Servlet and other Java web technologies. To make the matter worse, there is no HTTP or HTML library in core JDK as well; or at least I am not aware of that. That's why when it comes to parse a HTML file, many Java programmers had to look at Google to find out how to get value of an HTML tag in Java. When I needed that I was sure that there would be an open source library which will does it for me, but didn't know that it was as wonderful and feature rich as JSoup. It not only provides support to read and parse HTML document but also allows you to extract any element form HTML file, their attribute, their CSS class in JQuery style and also allows you to modify them. You can probably do anything with HTML document using Jsoup. In this article, we will parse and HTML file and find out value of title and heading tags. We will also see example of downloading and parsing HTML from file as well as any URL or internet by parsing Google's home page in Java.
What is JSoup Library
Jsoup is an open source Java library for working with real-world HTML. It provides a very convenient API for extracting and manipulating data, using the best of DOM, CSS, and jquery-like methods. Jsoup implements the WHATWG HTML5 specification, and parses HTML to the same DOM as modern browsers like Chrome and Firefox do. Here are some of the useful features of jsoup library :
- Jsoup can scrape and parse HTML from a URL, file, or string
- Jsoup can find and extract data, using DOM traversal or CSS selectors
- Jsoup allows you to manipulate the HTML elements, attributes, and text
- Jsoup provides clean user-submitted content against a safe white-list, to prevent XSS attacks
- Jsoup also output tidy HTML
Jsoup is designed to deal with different kinds of HTML found in the real world, which includes proper validated HTML to incomplete non-validate tag collection. One of the core strength of Jsoup is that it's very robust.
HTML Parsing in Java using JSoup
In this Java HTML parsing tutorial, we will see three different example of parsing and traversing HTML document in Java using jsoup. In first example, we will parse an HTML String which contents all tags in form of String literal in Java. In Second example, we will download our HTML document from web, and in third example, we will load our own sample HTML file login.html for parsing. This file is a sample HTML document which contains title tag and a div in body which contains an HTML form. It has input tags to capture username and password and submit and reset button for further action. It's proper HTML which can be validated i.e. all tags and attributes are properly closed. Here is how our sample HTML file look like :
DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Login Pagetitle>
head>
<body>
<div id="login" class="simple" >
<form action="login.do">
Username : <input id="username" type="text" /><br>
Password : <input id="password" type="password" /><br>
<input id="submit" type="submit" />
<input id="reset" type="reset" />
form>
div>
body>
html>HTML parsing is very simple with Jsoup, all you need to call is static method Jsoup.parse() and pass your HTML String to it. JSoup provides several overloaded parse() method to read HTML file from String, a File, from a base URI, from an URL, and from an InputStream. You can also specify character encoding to read HTML files correctly which is not in "UTF-8" format. Here is complete list of HTML parse method from JSoup library. The parse(String html) method parses the input HTML into a new Document. In Jsoup, Document extends Element which extends Node. Also TextNode extends Node. As long as you pass in a non-null string, you're guaranteed to have a successful, sensible parse, with a Document containing (at least) a head and a body element. Once you have a Document, you can get the data you want by calling appropriate methods in Document and its parent classes Element and Node.
Java Program to parse HTML Document
Here is our complete Java program to parse an HTML String, an HTML file download from internet and an HTML file from local file system. In order to run this program, you can either use Eclipse IDE or you can just use any IDE or command prompt. In Eclipse, it's very easy, just copy this code, create a new Java project, right click on src package and paste it. Eclipse will take care of creating proper package and Java source file with same name, so absolutely less work. If you already have a Sample Java project, then it's just one step. Following Java program shows 3 examples of parsing and traversing HTML file. In first example, we directly parse an String with html content, in second example we parse an HTML file downloaded from an URL, in third example we load and parse an HTML document from local file system. In first and third example we use parse method to get a Document object which can be queried to extract any tag value or attribute value. In second example, we use Jsoup.connect() with, which takes care of making connection to URL, downloading HTML and parsing it. This method also returns Document object which can be used for further querying and getting value of any tag or attribute.
import java.io.File;
import java.io.IOException;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
/**
* Java Program to parse/read HTML documents from File using Jsoup library.
* Jsoup is an open source library which allows Java developer to parse HTML
* files and extract elements, manipulate data, change style using DOM, CSS and
* JQuery like method.
*
* @author Javin Paul
*/
public class HTMLParser{
public static void main(String args[]) {
// Parse HTML String using JSoup library
String HTMLSTring = ""
+ ""
+ ""
+ "JSoup Example "
+ ""
+ ""
+ ""
" HelloWorld
+ "
+ ""
+ "";
Document html = Jsoup.parse(HTMLSTring);
String title = html.title();
String h1 = html.body().getElementsByTag("h1").text();
System.out.println("Input HTML String to JSoup :" + HTMLSTring);
System.out.println("After parsing, Title : " + title);
System.out.println("Afte parsing, Heading : " + h1);
// JSoup Example 2 - Reading HTML page from URL
Document doc;
try {
doc = Jsoup.connect("http://google.com/").get();
title = doc.title();
} catch (IOException e) {
e.printStackTrace();
}
System.out.println("Jsoup Can read HTML page from URL, title : " + title);
// JSoup Example 3 - Parsing an HTML file in Java
//Document htmlFile = Jsoup.parse("login.html", "ISO-8859-1"); // wrong
Document htmlFile = null;
try {
htmlFile = Jsoup.parse(new File("login.html"), "ISO-8859-1");
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} // right
title = htmlFile.title();
Element div = htmlFile.getElementById("login");
String cssClass = div.className(); // getting class form HTML element
System.out.println("Jsoup can also parse HTML file directly");
System.out.println("title : " + title);
System.out.println("class of div tag : " + cssClass);
}
}
Output:
Input HTML String to JSoup :DOCTYPE html><html><head><title>JSoup Exampletitle>head><body><table><tr><td><h1>HelloWorldh1>tr>table>body>html>
After parsing, Title : JSoup Example
Afte parsing, Heading : HelloWorld
Jsoup Can read HTML page from URL, title : Google
Jsoup can also parse HTML file directly
title : Login Page
class of div tag : simple
Good thing about JSoup is that it is very robust. Jsoup HTML parser will make every attempt to create a clean parse from the HTML you provide, regardless of whether the HTML is well-formed or not. It can handle following mistakes :
unclosed tags (e.g. <p>Java <p>Scala to <p>Javap> <p>Scalap>)
implicit tags (e.g. a naked <td>Java is Greattd> is wrapped into a <table><tr><td>)
reliably creating the document structure (html containing a head and body, and only appropriate elements within the head)
That's all about how to parse an HTML document in Java. Jsoup is an excellent and robust open source library which makes reading html document, body fragment, html string and directly parsing html content from web extremely easy. In this article, we learned hot to get value of a particular html tag in Java, as in fist example we extracted title and value of H1 tag as text, and in third example we learned how to get value of an attribute from html tag by extracting CSS class. Apart from powerful jQuery style html.body().getElementsByTag("h1").text() method, which you can use to extract any HTML tag, it also provides convenience methods like Document.title() and Element.className() method to quickly get title and CSS class. Have fun with Jsoup and we will see couple of more examples of this API soon.
Tidak ada komentar:
Posting Komentar