Wednesday, April 23, 2014

Thread Pools in Java

To scale applications it becomes necessary to separate thread creation and management from rest of the application. Using thread pools is an approach typically employed in large scale systems. This article provides a quick overview on executor framework in Java and provides examples on how to work with thread pools.

Significance of Thread Pools

Thread Pools use worker threads to minimize thread creation overhead.
Thread Pools reduce the memory management overhead which is important for large scale applications.
Thread Pools allow the applications to degrade gracefully.

Executor Service Objects

Java supports executor framework which provide the enablers for thread creation and management. Listed are some of the Java objects of executor service and their key purpose.

Executor

Executor interface provides a way to decouple tasks from how a task will run, thread to use etc. Executor is normally used instead of creating threads.

ExecutorService

ExecutorService interface represents an asynchronous execution mechanism which is capable of executing tasks in the background.

ThreadPoolExecutor

ExecutorService is an interface and ThreadPoolExecutor is one of the concrete implementations of ExecutorService. Executes submitted tasks using one of the several pooled threads. ThreadPoolExecutor provides many adjustable parameters and hooks. For ease of programming it is recommended to use the static factory methods provided by Executors.

Executors

Provides factory and utility methods for executor. They provide static methods like newFixedThreadPool(), newCachedThreadPool() which are much easier to use.

Approaches for Thread Pools

Java supports several approaches for handling thread pools. These include:

Fixed thread pool

This approach reuses fixed number of threads. At any point at-most "n" threads would be active. If additional tasks are submitted when all threads are active the tasks would be queued. Threads in the pool will exist until it is explicitly shutdown.

Cached thread pool

This approach creates new threads as needed, but will reuse previously constructed threads when they are available. If no threads are available for a task a new thread will be created and added to the pool. Threads that are not used for 60 secs will be terminated and removed from the cache. 

Single thread pool

This approach uses a single worker thread. Tasks submitted would be executed sequentially. This can be assumed equivalent to a fixed thread pool of size "1". The primary difference is that fixed thread pool can reconfigured to use additional threads but single thread pool is not re-configurable.

Example of Thread Pool usage

This example shows usage of fixed, cached and single thread pool.
package com.sourcetricks.threadpool;

import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;

public class ThreadPool {
 
 class MyThread extends Thread {

  private int id;
  
  public MyThread(int id) {
   this.id = id;
  }
  
  public void run() {
   System.out.println("Starting thread " + id);
   doSomeWork();
   System.out.println("Completed thread " + id);
  }
  
  private void doSomeWork() {
   try {
    Thread.sleep(5000);
   } catch (InterruptedException e) {
    e.printStackTrace();
   }
  }
 }
 
 // Not using thread pool
 private void doWithoutThreadPool() {
  for ( int i = 0; i < 20; i++ ) {
   MyThread thread = new MyThread(i);
   thread.start();
  }
 }
 
 // Using fixed thread pool
 private void doWithFixedThreadPool1() throws InterruptedException {
  ExecutorService executor = Executors.newFixedThreadPool(5);
  for ( int i = 0; i < 20; i++ ) {
   MyThread thread = new MyThread(i);
   executor.execute(thread);
  }
  System.out.println("Active thread count = " + ((ThreadPoolExecutor)executor).getActiveCount());
  
  // Don't accept new work
  executor.shutdown();
  
  // Wait for 30 secs for the threads to complete
  executor.awaitTermination(30, TimeUnit.SECONDS);
 }
 
 // Using fixed thread pool. Directly uses the ThreadPoolEexcutor
 // which provides finer control
 private void doWithFixedThreadPool2() throws InterruptedException {
  int core = 5;
  int max = 10;
  int keepalive = 5000;
  
  ExecutorService executor = new ThreadPoolExecutor(
    core, 
    max, 
    keepalive, 
    TimeUnit.MILLISECONDS, 
    new LinkedBlockingQueue<Runnable>());
  
  for ( int i = 0; i < 20; i++ ) {
   MyThread thread = new MyThread(i);
   executor.execute(thread);
  }
  
  System.out.println("Active thread count = " + ((ThreadPoolExecutor)executor).getActiveCount());
  executor.shutdown();
  executor.awaitTermination(30, TimeUnit.SECONDS);
 }
 
 // Using cached thread pool
 private void doWithCachedThreadPool() throws InterruptedException {
  ExecutorService executor = Executors.newCachedThreadPool();  
  for ( int i = 0; i < 20; i++ ) {
   MyThread thread = new MyThread(i);
   executor.execute(thread);
  }
  System.out.println("Active thread count = " + ((ThreadPoolExecutor)executor).getActiveCount());
 }
 
 // Using single thread pool
 private void doWithSingleThreadPool() throws InterruptedException {
  ExecutorService executor = Executors.newSingleThreadExecutor();
  // Executor returned is not reconfigurable
  for ( int i = 0; i < 20; i++ ) {
   MyThread thread = new MyThread(i);
   executor.execute(thread);
  }
 }

 public static void main(String[] args) {
  
  ThreadPool threadPool = new ThreadPool();
  try {
   threadPool.doWithCachedThreadPool();
  } catch (InterruptedException e) {
   e.printStackTrace();
  }
 }
}

Saturday, March 8, 2014

Java Serialization

Serialization is the process of converting the current state of an object into a byte stream. Deserialization is the process of converting the serialized form of an object back into a copy of the object.  A Java object is Serializable if a class implements java.io.Serializable interface. java.io.Serializable is a marker interface which tells that object is Serializable.

A quick example. In this example Employee class implements the marker interface Serializable and can be serialized. 
package com.sourcetricks.java;

import java.io.Serializable;

public class Employee implements Serializable {

 private static final long serialVersionUID = 2L;
 private int id;
 private String name;
 private int deptId;
 Employee() {   
 }
 Employee(int id, String name, int deptId) {
  this.id = id;
  this.name = name;
  this.deptId = deptId;
 }
 public int getId() {
  return id;
 }
 public String getName() {
  return name;
 }
 public int getDeptId() {
  return deptId;
 }
 public void setId(int id) {
  this.id = id;
 }
 public void setName(String name) {
  this.name = name;
 }
 public void setDeptId(int deptId) {
  this.deptId = deptId;
 }
 public void print() {
  System.out.println(id);
  System.out.println(name);
  System.out.println(deptId);
 }
}

In the test program we create an instance of Employee and using writeObject on ObjectOutputStream we persist the serialized object to the file system. During deserialization we do the reverse using readObject on ObjectInputStream to create a copy of the object back.
package com.sourcetricks.java;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;

public class JavaSerializationTest {
 
 public static void main (String[] args) {
  
  Employee e = new Employee(100, "John", 14);
  e.print();
  
  // Serialize and save the object to file
  FileOutputStream fout = null;
  ObjectOutputStream out = null;
  try {
   fout = new FileOutputStream("data.out");
   out = new ObjectOutputStream(fout);
   out.writeObject(e);
   out.close();
   fout.close();
  } catch (IOException e1 ) {
   e1.printStackTrace();
  }
  
  // Read object from file
  FileInputStream fin = null;
  ObjectInputStream in = null;
  try {
   fin = new FileInputStream("data.out");
   in = new ObjectInputStream(fin);
   Employee e1 = (Employee) in.readObject();
   e1.print();
   in.close();
   fin.close();
  } catch (IOException e1 ) {
   e1.printStackTrace();
  } catch (ClassNotFoundException e2) {
   e2.printStackTrace();
  }
 }
}
Output:
100
John
14
100
John
14

Sgnificance of serialVersionUID

serialVersionUID is important for classes to be serialized. It allows to specify a version to the object. Lets say object version is 1 when the object was written to the file system. At a later point of time there is a type change in the object and the version is upgraded to 2. If we try to deserialize the original object written to file system then it results in an InvalidClassException. 

java.io.InvalidClassException: com.sourcetricks.java.Employee; local class incompatible: stream classdesc serialVersionUID = 1, local class serialVersionUID = 2

Significance of transient

Sometimes it is necessary that one of the class members should not be included as part of the serialization process. Either it has sensitive information or which is not relevant once the object is de-serialized. (e.g. current time). In such scenarios associating transient keyword prevents the class member from being included as part of writeObject.

Saturday, February 22, 2014

FreeMarker Introduction

FreeMarker Introduction

FreeMarker is a template engine that helps to generate text based on templates. Examples, generation of dynamic web pages, source code generators etc. Freemarker helps to separate the application logic and page design. FreeMarker is a Java library and is free to use with the BSD license terms.

FreeMarker supports a powerful template language which helps to bind a data-model and template together to produce the output.

Data Model 

All the data that the template can use/ refer is packed into a data-model. From the perspective of the template the data-model is a tree structure holding data. The data model tree can contain scalars that store a single value, containers that store other variables or sequences that store other variables in an ordered sequence.

Template

Templates use FTL tags (Freemarker Template Language) tags that provide instructions to FreeMarker. All the Freemarker tags start with #. 

FreeMarker Download

To use the examples below download Freemarker from http://freemarker.org/ and include the freemarker.jar in the application CLASSPATH. There are no other dependencies.

Freemarker Example

This example is an introduction to get quickly started with FreeMarker and understand the basic usage principles. Let's say we are interested in reporting a list of books (preferably in database) as a HTML table.

Data Model Example

First step is to create the data model. For simplicity we have ignored reading the actual data from database but instead fill our data model with some static data. Here we create a data model with a list of books and a scalar variable with report generated date interpreted as string.

// Create a data-model      
Map<String, Object> root = new HashMap<String, Object>();
List<Object> bookList = new LinkedList<Object>();
bookList.add(new Book("Gambardella, Matthew", "XML Developer's Guide"));
bookList.add(new Book("Ralls, Kim", "Midnight Rain"));
bookList.add(new Book("Corets, Eva", "Maeve Ascendant"));
root.put("bookList", bookList);

Date now = new Date();
root.put("ReportDate", now.toString());

For reference here is the Book class we use.

package com.sourcetricks.freemarker;

public class Book {

    private String author;
    private String title;
 
    public Book(String author, String title) {
       this.author = author;
       this.title = title;
    }
 
    public String getAuthor() {
       return author;
    }
 
    public String getTitle() {
      return title;
    }
}

Template Example

In this template we use FTL tags to bind data in our model to the HTML page. We use <#if> to check if books are present and <#list> to parse the book list. 

<!DOCTYPE html>
<html>
<head>
  <title>Welcome!</title>
</head>
<body>
  <#if bookList??>
     <p>No of books = ${bookList?size}</p>
     <table border="1">
       <tr>
         <th>Author</th>
         <th>Title</th>
       </tr>
       <#list bookList as book>
       <tr>
         <td>${book.author}</td>
         <td>${book.title}</td>
       </tr>
       </#list>
      </table>
   <#else>
       No books found
   </#if>
   <p>Reported at: ${ReportDate}</p>
</body>
</html>  

Main Program

Here is our main program which binds the data model and the template together and produces the required HTML page output.

package com.sourcetricks.freemarker;

import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.Writer;
import java.util.Date;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;

import freemarker.template.Configuration;
import freemarker.template.DefaultObjectWrapper;
import freemarker.template.Template;
import freemarker.template.TemplateException;
import freemarker.template.TemplateExceptionHandler;
import freemarker.template.Version;

public class FreemarkerTest {

  public static void main (String[] args) {
  
    // Create and adjust the configuration. Needed once for the application.
    Configuration cfg = new Configuration();

    try {
      cfg.setDirectoryForTemplateLoading(new File("resources"));
    } catch (IOException e) {
      e.printStackTrace();
    }
    
    cfg.setObjectWrapper(new DefaultObjectWrapper());
    cfg.setDefaultEncoding("UTF-8");
    cfg.setTemplateExceptionHandler(TemplateExceptionHandler.HTML_DEBUG_HANDLER);
    cfg.setIncompatibleImprovements(new Version(2, 3, 20));
    
    // Create a data-model      
    Map<String, Object> root = new HashMap<String, Object>();
    List<Object> bookList = new LinkedList<Object>();
    bookList.add(new Book("Gambardella, Matthew", "XML Developer's Guide"));
    bookList.add(new Book("Ralls, Kim", "Midnight Rain"));
    bookList.add(new Book("Corets, Eva", "Maeve Ascendant"));
    root.put("bookList", bookList);
    
    Date now = new Date();
    root.put("ReportDate", now.toString());
        
    try {
      // Get the template from the configured directory
      Template temp = cfg.getTemplate("test.ftl");
   
      // Process the template to merge data model 
      Writer out = new FileWriter("output.html");
      temp.process(root, out);
    } catch (IOException e) {
      e.printStackTrace();
    } catch (TemplateException e) {
      e.printStackTrace();
    }        
  }
}

Output



Sunday, February 9, 2014

Log4j properties configuration examples

Refer to this article Using Log4j for debugging in Java for a quick introduction on Log4j. Log4j is a simple, reliable, fast and extensible open source logging and tracing API. Logging is an important part of any software development lifecycle and is the only way to debug in certain production systems where it is not feasible to use debuggers. Here we provide most commonly used properties configuration examples.

Log to console only

# Set root logger
log4j.rootLogger=DEBUG, console

# Add log messages to console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n

Log to file only

# Root logger option
log4j.rootLogger=DEBUG, file

# File appender
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p - %m%n
log4j.appender.file.File=myoutput.log
log4j.appender.file.MaxFileSize=100KB

Log to both console and file

# Root logger option
log4j.rootLogger=DEBUG, file, console

# Direct log messages to stdout
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n

# File appender
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n
log4j.appender.file.file=myoutput.log
log4j.appender.file.MaxFileSize=100KB

Log to both console and file and with different log levels

# Root logger option
log4j.rootLogger=TRACE, file, console

# Direct log messages to stdout
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n
log4j.appender.console.Threshold=TRACE

# File appender
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n
log4j.appender.file.file=myoutput.log
log4j.appender.file.MaxFileSize=100KB
log4j.appender.file.Threshold=DEBUG

Log to console and customize level based on package

# Set root logger
log4j.rootLogger=TRACE, console

# Add log messages to console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n
log4j.logger.com.sourcetricks=DEBUG

Using Log4j for debugging in Java

Log4j is a simple, reliable, fast and extensible open source logging and tracing API. Logging is an important part of any software development lifecycle and is the only way to debug in certain production systems where it is not feasible to use debuggers.

To use these examples download Log4j from https://logging.apache.org/log4j/1.2/.

Let us quickly start with something minimal and the then try to understand Log4j terminology and APIs.

Using Log4J BasicConfigurator to log debug messages to the console

In this example we create a static logger instance and initialize using a basic configurator which logs all debug messages to the console.

package com.sourcetricks.log4j;

import org.apache.log4j.BasicConfigurator;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;

public class Log4jTest {
  // Initialize logger for instance Log4jTest
  static Logger log  = Logger.getLogger(Log4jTest.class);
    
  public static void main (String[] args) {
    // Basic configurator to log debug messages to the console
    BasicConfigurator.configure();         
          
    // Add some log messages
    log.debug("This is a debug message");
    log.trace("This is a trace message");         
  }
}

Output
0 [main] DEBUG com.sourcetricks.log4j.Log4jTest  - This is a debug message

Using Log4J BasicConfigurator to log all messages to the console

In this example we set the log level to log all messages to the console. Only difference from the previous example is that we have set the log level to log all messages to the console. Please note that in the output we get both the log messages unlike the previous example.

package com.sourcetricks.log4j;

import org.apache.log4j.BasicConfigurator;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;

public class Log4jTest {
  // Initialize logger for instance Log4jTest
  static Logger log  = Logger.getLogger(Log4jTest.class);
    
  public static void main (String[] args) {
    // Set the level to log all messages
    log.setLevel(Level.ALL);

    // Basic configurator to log all messages to the console
    BasicConfigurator.configure();         
          
    // Add some log messages
    log.debug("This is a debug message");
    log.trace("This is a trace message");         
  }
}

Output
0 [main] DEBUG com.sourcetricks.log4j.Log4jTest  - This is a debug message
2 [main] TRACE com.sourcetricks.log4j.Log4jTest  - This is a trace message

With this basic examples let us jump into some terminology to understand Log4j better.

Log4J Terminology

Logger Hierarchy

In Log4j the loggers follow a hierarchy. There is always a root logger accessed using Logger.getRootLogger() and all other loggers are instantiated and accessed using Logger.getLogger(). Logger instances follow a named hierarchy. For example, com.sourcetricks is considered parent of com.sourcetricks.log4jtest.

Log Levels

Loggers can be assigned various levels including OFF, TRACE, DEBUG, INFO, WARN, ERROR, FATAL and ALL. By default if a logger is not assigned a level it inherits the level from the first non null level from the logger hierarchy towards the root logger.

Appenders

Appenders define the output destination for logs. Examples include console, file, syslog etc. Appenders also follow the logger hierarchy.

Layouts

Log4j allows the log output line to be customized as well. This is called layouts.
e.g) 0 [main] DEBUG com.sourcetricks.log4j.Log4jTest  - This is a debug message

Configuration

Log4j provides the capability and flexibility to fully customize the logging needs either via APIs, Java properties (key = value) format or using XML files. Normal convention to use configuration files instead of APIs to perform the Log4j customization.

With this basic introduction let us look at some more examples to understand better the various customization options.

Using Log4J PropertyConfigurator to log debug messages to the console

In this example we use a PropertyConfigurator and specify the properties file to log debug messages to the console. Additionally we use the layout customization.

package com.sourcetricks.log4j;

import org.apache.log4j.Logger;
import org.apache.log4j.PropertyConfigurator;

public class Log4jTest {

    // Initialize logger for instance Log4jTest
    static Logger log  = Logger.getLogger(Log4jTest.class);
	
    public static void main (String[] args) {	
	// Property configurator
	PropertyConfigurator.configure("resources/log4j.console");		
		
	// Add some log messages
	log.debug("This is a debug message");
	log.trace("This is a trace message");
    }
}

Properties file
# Set root logger
log4j.rootLogger=DEBUG, console
 
# Add log messages to console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.Target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c - %m%n

Output
2014-02-09 07:58:03 DEBUG com.sourcetricks.log4j.Log4jTest - This is a debug message

With this basic understanding now it is all about customizing the property files to meet our customization needs. Please refer to this article Log4j properties configuration examples on some sample Log4J property file examples.

Popular Posts

Powered by Blogger.

 

© 2013, 2014 Programming Tutorials by SourceTricks. All rights resevered. Designed by Templateism

Back To Top