Site icon NgDeveloper

Multithreading Program in Java with online shop scrap Example

Mirthbees
Multithreading Program in Java with online shop scrap Example

Multithreading Program in Java with online shop scrap Example:

Multithreading is a process of executing multiple threads at a time. In our below example, we are sending the book ISBN number to 3 different online shops to scrap and get the price details. Final scrapped price details are written as JSON data. We have implemented multithreading in the below program by implementing runnable interface.

This below programs sends an ISBN number to amazon, crossword and uread shops to get the price of the book.

Before going further,

All the below given scrapping logics are just for understanding and learning purpose only. If you are planning to use commercially reach respective shops for approval / read their privacy policy to understand the do’s and don’s to avoid legal problems.

Logic in Overview:

  1. Runnable interface implemented for the static class MyRunnable inside the Main controller class.
    2. List of online shop names are stored in the string array.
    3. Online shopping scrapping logics are written in ScrapBookShops.java. Each online shops are written in each individual methods.
    4. All the online shop scrap methods are called inside the forloop by iterating the string array and calling the static MyRunnable class.
    5. MyRunnable class delegates the flow to each shops to scrap and get the price details.
    6. Each online shop scrap method sends the data to formJson method to write the output in json format.

Requirement:

Note: If you do not want to write your output in JSON format, then simple json jar is not required.

BookPriceController.java:

This is the main controller class which creates the thread to continue the individual online shop scrap flow.

ExecutorService executor = Executors.newFixedThreadPool(MYTHREADS);
String[] hostList = { "uread", "amazon", "crossword" };
String isbn = "8173711461";
for (int i = 0; i < hostList.length; i++) {
String url = hostList[i];
Runnable worker = new MyRunnable(url, isbn);
executor.execute(worker);
}
executor.shutdown();
// Wait until all threads are finish
while (!executor.isTerminated()) {
}

Note: Above snippet is the core part of the program, if you want the complete source code then click the download button to download the full source code of the BookPriceController class.

ScrapBookShops.java:

This is the business class, which has scrapping logics for all the given online shops to fetch the pricing details of the given input ISBN number.

public JSONObject delegateToShop(String shop, String isbn) {
if (shop.equalsIgnoreCase("uread")) {
return getUreadPrice(isbn);
} else if (shop.equalsIgnoreCase("amazon")) {
return getAmazonPrice(isbn);
} else if (shop.equalsIgnoreCase("crossword")) {
return getCrossWordPrice(isbn);
}
return null;
}

public JSONObject getUreadPrice(String isbn) {
String baseUrl = "http://www.uread.com/search-books/" + isbn + "";
String uReadPrice = "";
Document doc;
JSONObject uReadJSON = new JSONObject();
try {
doc = Jsoup.connect(baseUrl).timeout(0).get();
String url = doc.baseUri();
String outOfStock = doc.select("div.stock-info")
.select("div.out-of-stock").text();
uReadPrice = doc.select("div.sale").text();
uReadJSON = formJSON("Uread", formatPrice(uReadPrice), url);
} catch (IOException e) {
e.printStackTrace();
}
return uReadJSON;
}

Note: Above snippet explains only the uread scrapping logic, if you want the complete source code of the other shops as well then click the download button to download the full source code of the ScrapBookShops class.

Things to Note:

As given in the starting of the program, before using the above program ensure you have completed the privacy policy of the respective shops and understand clearly about the do’s and don’s to avoid legal issues.

We have given the above programs only to explain about how scrapping and multi threading works together. It’s for 100% learning only.

Feel free to write your thoughts, comments in the below comments section.

Exit mobile version