TimeAndSizeRollingAppender, a very handy Log4J appender that supports rolling log files by time and size. I haven’t found one that meet my requirements in the built-in/extra appenders provided by log4j project. DailyRollingFileAppender is the closest appender I could live with, but there is no way to control size of the file when rolling, which is very useful when triaging application issues at debug level trace.

TimeAndSizeRollingAppender appender is pretty cool, loving it so far. Compression is built-in and super easy to configure. It has many features and this comparison covers those in detail. Here is a sample log4j.properties file that I ended up using:

Possibly Related Posts:


Starting in JDK 9 build 54, compiler support for private interface methods have been resurrected. This was one feature that was planned for Java 8 and did not make the final cut. It is now part of JEP 213. The primary motivation for this proposal is to enable code sharing between non abstract methods in an interface.

Interface evolution was always a tricky problem to solve in earlier versions of Java. You either need to maintain multiple versions of APIs which creates a technical debt over time or brute force your clients to accommodate new methods which breaks binary compatibility. The need to enhance an interface needed support in interface themselves instead of resorting to other options. This was fulfilled by default methods in interfaces in Java 8. Taking this further, private methods come in handy when we have a reusable block of code that default methods could benefit from. Latest Intellij 14.1 builds started to support JDK 9, so lets code an interface!.

An interface now can contain these:

– constants
– method signatures
– nested types
– default methods (since 1.8)
– static methods (since 1.8)
– private methods (since 1.9)
– private static methods (since 1.9)

Some of the restrictions of private methods include:
– private methods must contain method body
– combination of modifiers not allowed : abstract and private
– private methods in an interface does not disqualify it from being a @FunctionalInterface
– annotations types cannot declare private methods
– name clash not permitted

– cannot reduce visibility of private methods

Possibly Related Posts:


Java 8 will go GA in March 2014. java8_logo The first proposed final draft of Java 8 JSR 337 was released early this week. One of the main components of this JSR is Lambda Expressions for the language (JSR 337). This brings the functional constructs to the language, which will undeniably make Java a top contender among other functional peers (Scala, Groovy, Clojure) on the JVM for Java developers. Conciseness is only part of the story, check out how Java 8 programs start to look with Lambda expressions. While being concise for a casual reader, it clearly makes a case why thinking functional may benefit to Java developers in terms of the powerful constructs it offers. The conciseness does not take away readability of the Java program, instead it makes Java developers feel home and enables functional programming with ease, taking away the heavy lifting usually required for moving away from OO paradigm.

I stole the name “Java.next” from Neal Ford‘s excellent series of articles about functional languages on the JVM. The latest installment of the series was “Java.next: Functional coding styles”. While, the article takes on legacy Java, adding Java 8 to the mix would not hurt. So, let us take his example where you are given a list of names, some of which consist of a single character. You are asked to return the names in a comma-delimited string that contains no single-letter names, with each name capitalized. Here are the various flavors of the implementation taken from the article, while Scala, Groovy and Clojure has its functional version, Java’s implementation is imperative and old school.

Java version:

Scala version:

Groovy version:

Clojure version:

Happy Holidays from the islands

Java 8 version of functional processing

With Java 8, functional interfaces makes this possible with Lambda expressions and Streams make you even more fun when working with Collections in Java 8. Here is the functional version in Java 8.

Switch it to parallelStream() for parallel processing, as simple as that to kick the tires of your cores.

My original intent was to compare with the Java.next languages as perceived by Neal Ford in his article. It is unfair to leave behind some of the cool languages on the JVM. Thanks Luke for bringing this up. I strongly believe top-notch IDE support (Eclipse or Intellij) is a win-win situation for users and language implementors. With that context, I updated my list to include Fantom, Gosu, Xtend, Kotlin, Ceylon in order of their age (oldest first).

Fantom version:

Gosu version

Xtend version:

Kotlin version:

Ceylon version:

Personally, I feel Java 8 and Groovy has better success rate in making developers stay in their comfort zone and still enjoy functional prowess. While, I agree Clojure and Scala versions make their case to different set of developers, I believe maintaining a larger code base will be a challenge, when time to develop and market is ever shrinking, and bringing someone up-to-speed with these language tricks will not be easy. If it looks good, eat it, does not play well when it comes to programming in general. While coming to the newer JVM languages, parallelism is one factor that makes Java 8 easy peasy to deal with. These new breed of languages bring ton of cool features (topic for another blog post!) and attracts developers to the JVM, not just Java developers, which is good for the platform. It remains to be seen in the coming years how well these languages get adopted in mainstream development.

Merry Christmas and Happy New Year everyone!

Update (12/29/2013) : Updated the Java.next languages comparison to include newer JVM languages: Fantom, Gosu, Xtend, Kotlin, Ceylon.

Possibly Related Posts:


The default chunk size in Apache HttpClient 4.3 is set to 2048 bytes. There is no easy way to customize the chunk size in the latest Apache HttpClient release. It looks like the only approach is to implement a custom ClientConnectionFactory that allows to override the default chunk size. Here is an example.

Possibly Related Posts:


Two years ago, I blogged about parallel for each construct using a very early version of lambda prototype compiler. At that time, Data Parallelism was already available in C# TPL, but Java designers were hard at work in bringing this idiom to Java.

Lambda expressions support in Java 8 is here for everyone to give it a spin. Functional touch to Collections library makes it a breeze to program in Java. you get this one-liner using Java 8 Stream API.

There are more such one-liners you can think of using parallel constructs in Java 8. Here is another teaser, send an email blast to all customers.

Streams API brings several of the LINQ as well as PLINQ awesomeness to Java Collections library. Here is an adhoc query over a collection using the new Streams API.

Check it out here for more one-liners. You may want to checkout this article that compares LINQ with Java Streams API. While, there are constraints with the Java Streams proposal, it hits the right balance and there is a pedagogical value to it as well, keeping majority Java developers happy.

Enjoy!

Possibly Related Posts:


Java 8 adds support for bulk data operations (JEP 107) in the Collections library. It is unlikely every forms of bulk data operations will be added to the retrofitted Collections framework, but the language designers are making every possible effort in accommodating general use cases of “filter/map/reduce for Java”. Libraries should make use of parallelization on multi-core platforms in order to perform massive parallelizable operations and this means literally kicking-in your cores to its fullest potential. This area of OpenJDK is undergoing heavy changes, it is possible these APIs are changed or improved before milestone 6 when JDK 8 is officially feature complete. But, they are stable enough to give it a spin today. You can grab the lambda build from here. I have been using the latest Intellij 12 preview for my hands on with the new features in Java 8. It actually supports most of the new Java 8 languauge features and intellisense rocks.

The Streams proposal is well thought of and brings the much sought after functional power to Java and you won’t regret upgrading from Java 6 -> Java 8, if you are still hanging around with Java 6 in that time frame. This one is just a preview of what is coming in Java 8.

Let us implement a method to check if a keyword list exists in a tweet. Here is one way to implement it in plain old Java.

Now, that I am taking the Functional programming course, I would like to compare it with Scala, the mature functional language on the JVM. In Scala, this can be implemented as one liners and there are couple different ways to do this, one using exists and other using the foldLeft function, although exists is my favorite as it is clean and less cryptic.

I can understand why functional programmers are more obsessed with “small is beautiful” theme and hate outright verbosity. Now that all dust has settled with Java’s once uncertain future and stalled innovation, its new steward Oracle is betting on to bring back its lost glory in round 2 and you see that happening with Java 8. Finally, the language brings freshness to Java camp in offering functional constructs to Java and make functional programming in Java a breeze. Here is a teaser:

Isn’t nice to have a one liners in Java too? Java is close, but could reduce some noise as you have to call stream() on the Collection which might seem unnecessary. Stream interface supports a fold operation very similar to Scala’s foldLeft function and implements using a similar approach as of Scala.

Now, let us take this example further, filter tweets matching keywords. This was inspired by Week 3 assignment from the “Functional Programming Principles in Scala” course which btw I am enjoying it so far. Again, you can see Java makes it a cut. I am trying to keep it not being a spoiler.


In this example, you notice that you have to use stream() as well as into() operations to get the result of stream processing. I believe this is a limitation with the iteration 2 Streams implementation which was referred as “the bun problem” that we have to live with.

Streams in Java 8 is making implementing bulk operations a no brainer. While it involves some effort in learning the new concepts, it opens up a new dimension to library developers. Thanks to JSR 335 which brings Java up to speed with other cool languages out there in the planet.

For fun, I want to toss in a new exists() API to the Stream interface so I could use it to filter on my keywords list. This can be done with the help of extension methods, another new concept in Java 8.


Basically, I am delegating the implementation to anyMatch() for simplicity as both reveal the same purpose. Here is a custom List implementation that exposes this API on the collection. This is a primitive implementation, which btw I am trying to keep this updated with the recent library changes. I hope there is a better way to do this, but the point is, it enables developers to extend the Streams framework and fill in special cases.

exists() variant of the code is shown below.


There are several new concepts in the library that requires functional mindset to fully benefit from it. You can switch these Streams to use a parallel version, by using parallel() instead of stream() on the Collection. Stream operations are classified as lazy and eager based on their implementation.
I hope to uncover some of these new concepts in the coming months. I wish function types made into Java 8, but that is different story for another day. Enjoy functional programming in Java!

I have covered Lambdas in the past and this excellent JavaZone talk by Angelika Langer should bring you up to speed with the current state of affairs with Lambdas in Java 8. You can also find Maurice Naftalin’s LambdaFAQ useful.

Lambdas in Java 8 from JavaZone on Vimeo.

Update (10/20/2012) : Updated samples to use method reference and built with the latest Lambda build 61. Clarified the bun-problem with streams proposal.
Update (11/20/2012) : Updated MyList implementation to compile with the latest API changes and much cleaner implementation.
Update (2/3/2013) : Updated code to compile with the latest lambda build 75.
Update (6/15/2013) : Updated code to compile with the latest lambda build 94.

Possibly Related Posts:


Last November, Mark Reinhold announced that Closures will be added to the Java language and six months later we had an initial prototype based on the Straw-Man proposal and last month’s update was considered a significant progress for the language in years. As mentioned in Mark’s post, “Working with parallel arrays in Java, unfortunately, requires lots of boilerplate code to solve even simple problems. Closures can eliminate that boilerplate.” In my earlier post, I discussed how Lambda expressions make Aggregate operations a breeze to work with in Java. But, we still lack parallel looping constructs such as parallel for, foreach APIs that would take advantage of the multi-core machines we have today.

In this next Lambda adventure, I will be revisiting the Parallel ForEach example discussed by Howard in his “New Control Structures for Java” post, where he explains the use of anonymous inner classes to map a function that executes in parallel and retains the order of the original map, handles exceptions and implements continue operation. Here is the anonymous inner class version of the code:

The program produces the below output:

{1=H, 2=e, 3=l, 4=l, 5=o}

Now, that we have a working Lambda prototype, this code could be modified to take advantage of Lambda expressions. Let us enable the Lambda finder option in the compiler that identifies potential candidates for converting an anonymous inner class instance to an Lambda expression. Here is the updated javac.bat used to run with the latest prototype compiler.

@echo off
C:\jdk1.7.0\bin\java.exe -cp classes.jar com.sun.tools.javac.Main -XDidentifyLambdaCandidate=true %1

Compiling Parallel.java with this hidden compiler option identifies 2 candidates for Lambda expressions.

C:\JDK7\lambda\samples>javac.bat Parallel.java
Parallel.java:16: Note: This anonymous inner class creation can be turned into a lambda expression.
            return new Callable<O>() {
                                     ^
Parallel.java:70: Note: This anonymous inner class creation can be turned into a lambda expression.
        final ForEach<Character, Integer, Character> trim = new ForEach<Character, Integer, Character>() {
C:\JDK7\lambda\samples>

Let us quickly refactor Howard’s code using Lambda expressions and here is my take.

This version does not significantly change from its anonymous counterpart, it reduces at most 10 lines of code. But, it introduces two important concepts in addition to use of Lambda expressions. Lines 6 and 13 uses the new exception transparency mechanism and Line 62 shows how lexical scoping makes Lambda expressions different from anonymous inner classes.

Exception Transparency

Exception transparency is a new feature proposed as part of the umbrella Lambda proposal. While this feature is not required to support Lambda expressions, it is useful in library implementations that require abstraction over generic exceptions. Java Generics does a decent job at abstracting over method return types and argument types, leaving abstraction over types of checked exceptions in limbo. This makes it difficult for libraries to support generic exception types in APIs. The two extreme cases where libraries usually model their exception types are :

In case of Runnable, the run() method throws nothing.

public interface Runnable {
    public abstract void run();
}

In case of Callable, the call() method throws everything.

public interface Callable<V> {
    V call() throws Exception;
}

As more and more libraries would take advantage of executing Lambda expressions as a block of code, the inability to generify thrown exception types makes this proposal a compelling use case.

A throws type parameter is a generic type parameter that is introduced with the keyword throws; throws type parameters implicitly have an upper bound of Exception (though any upper bound that is a subtype of Throwable may be explicitly specified) and can correspond to zero or more actual types that are subtypes of the upper bound.

With this support in the language, we should be able to abstract generic exception types:

	interface Block<throws E> {
		public void execute() throws E;
	}

Lexical scoping

Unlike anonymous inner classes, lambda expressions are lexically scoped. This means that code inside the Lambda body sees only what is available immediately outside the lambda expression. For instance, access to names from inherited classes/interfaces is not allowed as seen in Line #62 of our example above.

I wanted to briefly touch upon these concepts that are relevant to our discussion. Please refer to the proposal for more details.

.NET world

Let us look at how modern languages support Parallelism in their libraries. Here is an closest example using C# Task Parallel Library (TPL) (from Patterns of Parallel Programming Guide).

But this is just an attempt if someone wanted to write their own parallel foreach construct using existing C# APIs. But, this is not required. In C#, parallel constructs are available for a while now. Using the Parallel.ForEach construct in TPL, iterating all the Test records of a Student and calculate the GPA would look something like:

Fair enough, this construct is concise and abstracts all the machinery from the Client code. Are we close enough to get a similar construct in Java? I think so, we do have parallel collections library that supports parallelism using Fork-Join framework today. It looks like Doug Lea has recently revealed his plans for developing new library components for parallel operations on aggregates (collections and arrays) for JDK8 (based on proposed language support for lambdas). I believe this effort would produce these constructs in Java.

For the sake of fun, I compiled the extry166y sources with the compiler option “-XDidentifyLambdaCandidate=true”, it identified 168 candidates that can be turned into a lambda expression. Less code means more fun and you see that for real.

Update (6/11/2011) : Updated the Parallel class (Lambda version) to compile with the latest lambda prototype compiler. Abstract classes with single abstract method are no longer considered valid SAM types as per the revised rules.

Possibly Related Posts:


There are two features that are very cool in the recent CXF 2.3 release that deserves its mention in the release notes/features document as they prove to be quite useful and powerful in certain use cases. Advanced search capabilities and Atom Logging features make CXF a compelling choice for developers looking for this support in JAX-RS frameworks. I first read about these features when Sergey introduced their availability in 2.3 snapshots.

In this post, I am going to explore FIQL based advanced search query support in CXF. I have experimented Atom logger before and it deserves a separate blog post and when used with FIQL, they are more powerful.

Introduction to FIQL Expressions in CXF

The Feed Item Query Language (FIQL, pronounced “fickle”) is a draft specification by Mark Nottingham that proposes URL-friendly syntax for expressing filters that operate on the feed entries. An FIQL expression is composed of one or more constraints, related to each other with Boolean operators. FIQL expressions yield Boolean values: True or False.

Let us look at each of the operators defined in the draft spec with an example FIQL expression and the equivalent SQL query produced by CXF.

Two boolean operators:
“;” is the Boolean AND operator; it yields True for a particular entry if both operands evaluate to True, otherwise False.

/sakila/searchActors?_s=firstname==PENELOPE;lastname==GUINESS
SELECT * FROM actor WHERE lastname = 'GUINESS' AND firstname = 'PENELOPE'

“,” is the Boolean OR operator; it yields True if either operand evaluates to True, otherwise False.

/sakila/searchActors?_s=lastname==MONROE,lastname==GUINESS
SELECT * FROM actor WHERE (lastname = 'MONROE') OR (lastname = 'GUINESS')

Two comparison operators are applicable to simple text comparisons:
“==” yields True if the string value (as per XPath) of any selected node matches the argument; otherwise False.

/sakila/searchActors?_s=lastname==MONROE
SELECT * FROM actor WHERE lastname = 'MONROE'

“!=” yields True if the string value of every selected node does not match the argument; otherwise False.

/sakila/searchActors?_s=lastname!=MONROE
SELECT * FROM actor WHERE lastname <> 'MONROE'

If the argument string begins or ends with an asterisk character (“*”), it acts as a wild card, matching any characters preceding or following (respectively) that position.

/sakila/searchActors?_s=lastname==PEN*
SELECT * FROM actor WHERE firstname LIKE 'PEN%'

[Note: The spec mentions that text comparisons should allow case insensitive textual content. CXF text comparison is case sensitive.]

Four operators are relevant to date comparisons:
“==” yields True if the point in time specified in the argument matches that indicated by the string-value of the selected node; otherwise False.

/sakila/searchRentals?_s=returndate==2005-08-31T20:00:25.000%2B06:00
SELECT * FROM rental WHERE returndate = 'Wed Aug 31 08:00:25 MDT 2005'

[Note: The database contains the following timestamp for the returndate ‘2005-08-31T20:00:25-06:00’, but when I query this using the above FIQL expression, it returned no matching entry. I tried another similar expression with no success: /sakila/searchRentals?_s=returndate==2005-08-31T20:00:25Z]

“!=” yields True if the point in time specified in the argument does not match that indicated by the string-value of the selected node; otherwise False.

/sakila/searchRentals?_s=returndate!=2005-08-31T20:00:25.000%2B06:00
SELECT * FROM rental WHERE returndate <> 'Wed Aug 31 08:00:25 MDT 2005'

“=lt=” yields True if the point in time specified in the argument follows that indicated by the string-value of the selected node; otherwise False.

/sakila/searchRentals?_s=rentaldate=lt=2005-05-27T00:00:00.000%2B00:00
SELECT * FROM rental WHERE rentaldate < 'Thu May 26 18:00:00 MDT 2005'

“=le=” yields True if the point in time specified in the argument follows that indicated by the string-value of the selected node, or is equal to it; otherwise False.

/sakila/searchRentals?_s=returndate=le=2005-05-27T00:00:00.000%2B00:00
SELECT * FROM rental WHERE returndate <= 'Thu May 26 18:00:00 MDT 2005'

“=gt=” yields True if the point in time specified in the argument precedes that indicated by the string-value of the selected node; otherwise False.

/sakila/searchRentals?_s=returndate=gt=2005-08-30T00:00:00.000%2B00:00
SELECT * FROM rental WHERE returndate > 'Mon Aug 29 18:00:00 MDT 2005'

“=ge=” yields True if the point in time specified in the argument precedes that indicated by the string-value of the selected node, or is equal to it; otherwise False.

/sakila/searchRentals?_s=returndate=ge=2005-09-01T00:00:00.000%2B00:00
SELECT * FROM rental WHERE returndate >= 'Wed Aug 31 18:00:00 MDT 2005'

Some advanced FIQL expressions prove to be very powerful when dealing with date range and relative comparison:

/sakila/searchRentals?_s=returndate=gt=%2DP5Y2M
SELECT * FROM rental WHERE returndate > 'Wed Aug 17 14:53:44 MDT 2005'
 (assuming processing on Oct 17th, 2010)

/sakila/searchRentals?_s=returndate=ge=2005-05-27T00:00:00.000%2B00:00;rentaldate=le=2005-06-27T00:00:00.000%2B00:00
SELECT * FROM rental WHERE rentaldate <= 'Sun Jun 26 18:00:00 MDT 2005' AND returndate >= 'Thu May 26 18:00:00 MDT 2005'

Four operators are relevant to numeric comparisons:

“==” yields True if the string-value of the selected node is numerically equal to the argument; otherwise False.

/sakila/searchFilms?_s=filmid==1;rentalduration!=0
SELECT * FROM film WHERE filmid = '1' AND rentalduration <> '0'

“!=” yields True if the string-value of the selected node is not numerically equal to the argument; otherwise False.

/sakila/searchFilms?_s=filmid!=1;rentalduration!=0
SELECT * FROM film WHERE filmid <> '1' AND rentalduration <> '0'

“=lt=” yields True if the string-value of the selected node evaluates as numerically less than the argument; otherwise, False.

/sakila/searchFilms?_s=filmid=lt=2;rentalduration!=0
SELECT * FROM film WHERE filmid < '2' AND rentalduration <> '0'

“=le=” yields True if the string-value of the selected node evaluates as numerically less than the argument, or as equal to it; otherwise, False.

/sakila/searchFilms?_s=filmid=le=2;rentalduration!=0
SELECT * FROM film WHERE filmid <= '2' AND rentalduration <> '0'

“=gt=” yields True if the string-value of the selected node evaluates as numerically greater than the argument; otherwise, False.

/sakila/searchFilms?_s=filmid=gt=995;rentalduration!=0
SELECT * FROM film WHERE filmid > '995' AND rentalduration <> '0'

“=ge=” yields True if the string-value of the selected node evaluates as numerically greater than the argument, or as equal to it; otherwise, False.

/sakila/searchFilms?_s=filmid=ge=995;rentalduration!=0
SELECT * FROM film WHERE filmid >= '995' AND rentalduration <> '0'

I believe this covers majority of the FIQL operators used in various proportions as documented in the spec and these FIQL expressions are not just examples, all of these searches can be tested with our sample Sakila database.

Sakila Database

Sakila is a MySQL sample database that represents a DVD rental store and comes with some useful data. This is handy for demonstrations and the schema is well designed. I will be using this sample database to demonstrate the advanced search capabilities using CXF. Java IDEs provide a decent support to reverse engineer JPA entities from a database schema. Here is one which walks through creating JPA entities using a NetBeans IDE from the Sakila database. I will be using these generated entities in this example with Hibernate as the JPA provider. The entire example is available as a maven project in Git, so feel free to check this out.

The meat of this example lies in the SakilaResource implemented as a CXF JAX-RS endpoint. In this code, a SearchContext is injected to the resource which will be used in the resource methods to get the SearchCondition specified in the query. FiqlParser does the magic for you in parsing an FIQL expression to construct these search conditions.

@Path("/sakila")
public class SakilaResource {

    @Context
    private SearchContext searchContext;

    List<Actor> actors = new ArrayList<Actor>();
    List<Film> films = new ArrayList<Film>();
    List<Rental> rentals = new ArrayList<Rental>();

    public SakilaResource() {
       // JPA Plumbing omitted for clarity.
    }

    @GET
    @Produces("application/xml")
    @Path("searchActors")
    public List<Actor> searchActors() {
        SearchCondition<Actor> sc = searchContext.getCondition(Actor.class);

        if (sc == null) {
            throw new NotFoundException("Invalid search query.");
        }
        System.out.println(sc.toSQL("actor"));

        List<Actor> found = sc.findAll(actors);
        if (found.size() == 0) {
            throw new NotFoundException("No matching actor found.");
        }
        return found;
    }

    @GET
    @Produces("application/xml")
    @Path("searchFilms")
    public List<Film> searchFilms() {
        SearchCondition<Film> sc = searchContext.getCondition(Film.class);

        if (sc == null) {
            throw new NotFoundException("Invalid search query.");
        }
        System.out.println(sc.toSQL("film"));

        List<Film> found = sc.findAll(films);
        if (found.size() == 0) {
            throw new NotFoundException("No matching film found.");
        }
        return found;
    }

    @GET
    @Produces("application/xml")
    @Path("searchRentals")
    public List<Rental> searchRentals() {
        SearchCondition<Rental> sc = searchContext.getCondition(Rental.class);

        if (sc == null) {
            throw new NotFoundException("Invalid search query.");
        }
        System.out.println(sc.toSQL("rental"));

        List<Rental> found = sc.findAll(rentals);
        if (found.size() == 0) {
            throw new NotFoundException("No matching rental found.");
        }
        return found;
    }
}

One can think of FIQL based search support in CXF is somewhat similar to using Hibernate Search to query JPA entities. I think Hibernate Search is a heavy weight solution which uses Lucene as the search provider and it requires annotating your entities. FIQL support in CXF is light weight and implementing advanced search capabilities in resources is a child’s play. Unlike Hibernate Search, CXF FIQL cannot be used to perform joins on multiple entities as the FIQL spec was designed for syndication feed data model.

In CXF, there are couple options to test these search conditions and they are easy to develop. One approach is to use the search extension API. You can find an example in the javadoc of SimpleSearchCondition.

The other approach is the traditional CXF way of using HTTP centric WebClient.

public class SakilaSearchTest {

    static Server server;

    @BeforeClass
    public static void setUp() {
        JAXRSServerFactoryBean sf = new JAXRSServerFactoryBean();
        sf.setResourceClasses(SakilaResource.class);
        sf.getInInterceptors().add(new LoggingInInterceptor());
        sf.getOutInterceptors().add(new LoggingOutInterceptor());
        sf.setResourceProvider(SakilaResource.class, new SingletonResourceProvider(new SakilaResource()));
        sf.setAddress("http://localhost:9000");
        server = sf.create();
    }

    @Test
    public void searchActors() {
        //http://localhost:9000/sakila/searchActors?_s=firstname==PENELOPE
        WebClient wc = WebClient.create("http://localhost:9000/sakila/searchActors?_s=firstname%3D%3DPENELOPE");
        Collection<? extends Actor> actors = wc.getCollection(Actor.class);
        assertEquals(4, actors.size());
    }

    @Test
    public void searchFilms() {
        //http://localhost:9000/sakila/searchFilms?_s=rating==PG;rentalduration!=0;title==SANTA*
        WebClient wc = WebClient.create("http://localhost:9000/sakila/searchFilms?_s=rating%3D%3DPG;rentalduration%21%3D0;title%3D%3DSANTA*");
        Collection<? extends Film> films = wc.getCollection(Film.class);
        assertEquals(1, films.size());
    }

    @Test
    public void searchRentals() {
        //http://localhost:9000/sakila/searchRentals?_s=rentaldate=lt=2005-05-27T00:00:00.000%2B00:00
        WebClient wc = WebClient.create("http://localhost:9000/sakila/searchRentals?_s=rentaldate%3Dlt%3D2005-05-27T00:00:00.000%2B00:00");
        Collection<? extends Rental> rentals = wc.getCollection(Rental.class);
        assertEquals(278, rentals.size());
    }

    @AfterClass
    public static void tearDown() {
        server.destroy();
    }
}

One thing I noticed with WebClient is that I need to encode certain operators specified in the search condition and this is inconsistent when testing with a web browser. I would expect the java API should take care of this for you instead of providing partially encoded URLs. Also, the date comparison can be tricky, although it works for most cases discussed above.

Possibly Related Posts:


Project Coin includes the following proposals that will make into Java 7.

Binary literals and underscores in literals

Strings in switch

– Simplified Varargs Method Invocation

– Improved Type Inference for Generic Instance Creation (Diamond)

– Improved Exception Handling for Java

– try-with-resources (ARM)

I have briefly touched upon try-with-resources construct in my earlier post. ARM was implemented in July and JDBC APIs were recently retrofitted to support these changes. In this installment, I will give you an overview of other Coin proposals.

Binary literals and underscores in literals

This feature is again we are trying to catch up with languages such as Ruby and Python where this is already supported.

You need to prefix the literal with 0b or 0B. Now, we can define binary literals in our code as show below.

int maxInteger = 0b01111111111111111111111111111111;// Integer.toBinaryString(2147483647)
byte nybbles = 0b00100101;// Math.pow(2, 5)+Math.pow(2, 2)+Math.pow(2, 0) = 37

Now, this can be better represented visually using underscores, which is addressed in the supplementary proposal:

int maxInteger = 0b0111_1111_1111_1111_1111_1111_1111_1111;// Integer.toBinaryString(2147483647)
byte nybbles = 0b0010_0101;// Math.pow(2, 5)+Math.pow(2, 2)+Math.pow(2, 0) = 37

Some more common usage of this feature,

int phoneNumber = 123_456_7890;
long creditCardNumber = 1234_5678_9012_3456L;
int socialSecurityNumber = 999_99_9999;
int hexEquivalentOfIntegerMAX = 0x7f_ff_ff_ff;// Integer.toHexString(2147483647)
int hexBytes = 0xFF_EC_DE_5E;// Integer.toHexString(-1253794)
int hexWords = 0xFFEC_DE5E;// Integer.toHexString(-1253794)
long maxLong = 0x7fff_ffff_ffff_ffffL;// Long.toHexString(9223372036854775807L)
int weirdBitfields = 0b000_10_101;// Integer.toBinaryString(21)

System.out.println("The value 10100001 in decimal is " + 0B10100001);// 161

More advanced example (from the proposal), do you see the happy face in bitmap?

// Binary literals can be used to make a bitmap more readable:
public static final short[] HAPPY_FACE = {
       (short)0b0000011111100000,
       (short)0b0000100000010000,
       (short)0b0001000000001000,
       (short)0b0010000000000100,
       (short)0b0100000000000010,
       (short)0b1000011001100001,
       (short)0b1000011001100001,
       (short)0b1000000000000001,
       (short)0b1000000000000001,
       (short)0b1001000000001001,
       (short)0b1000100000010001,
       (short)0b0100011111100010,
       (short)0b0010000000000100,
       (short)0b0001000000001000,
       (short)0b0000100000010000,
       (short)0b0000011111100000
};

This feature was implemented more than a year ago.

Strings in switch

We are familiar with the purpose of switch and what types are allowed in them.

A simple example from the Joe Darcy’s JavaOne Coin presentation:

    GregorianCalendar c = new GregorianCalendar();
    int monthNameToDays(String s, int year) {
        switch (s) {
            case "April":
            case "June":
            case "September":
            case "November":
                return 30;
            case "January":
            case "March":
            case "May":
            case "July":
            case "August":
            case "December":
                return 31;
            case "February":
                int days = 28;
                days += c.isLeapYear(year) ? 1 : 0;
                return days;
            default:
                return -1;
        }
    }

Here is a more complex one with nested switch :

public class SwitchTest {
    private static class Member {
        int id;
        String type;
        String category;

        private Member(int id, String type, String category) {
            this.id = id;
            this.type = type;
            this.category = category;
        }
    }

    public static int getMemberFee(Member member) {
        int fee = 50;
        switch (member.type) {
            case "BUSINESS":
                switch (member.category) {
                    case "LICENSED":
                    case "GOVERNMENT":
                    case "FARMERS":
                    case "RANCHERS":
                        break;

                    case "NON_PROFIT":
                        fee = 25;
                }
            case "GOLD_STAR":
                break;
            case "EXECUTIVE":
                fee = 100;
                break;
        }
        return fee;
    }

    public static int getMemberDiscount(Member member) {
        int discount = 5;
        switch (member.type) {
            case "BUSINESS":
                discount = 10;
                //fall through
            case "GOLD_STAR":
                break;
            case "EXECUTIVE":
                discount = 15;
                break;
        }
        return discount;
    }

    public static void main(String... args) {
        Member member = new Member(1234_5678_90, "BUSINESS", "NON_PROFIT");
        System.out.println("$" + SwitchTest.getMemberFee(member) + " fee for " + member.category);
        System.out.println(SwitchTest.getMemberDiscount(member) + "% discount for " + member.category);
    }
}

This feature was implemented almost a year ago.

Simplified Varargs Method Invocation

Before going into the details of this proposal, I would like to provide some background so it helps to understand the problem better. One of the well known restrictions introduced by Generics is its inability to create generic arrays. For example, the following code would not compile in Java:

    public static List<String>[] mergeLists() {
        List<String> a = Arrays.asList("one", "two", "three");
        List<String> b = Arrays.asList("four", "five", "six");
        return new List<String>[]{a, b};  // compile-time error: generic array creation
    }

For instance, let us compile the above code using Java 6 compiler using the printflat hidden option

javac -d /tmp -XD-printflat MergeLists.java

The generated file in /tmp would look something like:

    public static List[] mergeLists() {
        List a = Arrays.asList(new String[]{"one", "two", "three"});// type information is lost here
        List b = Arrays.asList(new String[]{"four", "five", "six"});// type information is lost here
        return List[] {a, b};// added for completion, this is illegal.
    }

As a consequence of Generics, type information is erased after compilation, making it unavailable at run time. So, types such as E, List, and List are called non-reifiable types [JLS 4.7]. Varargs are implemented as an array of objects and can hold only reifiable types.

When a programmer tries to invoke a varargs method with non-reifiable varargs type, the compiler currently generates an “unchecked” warning. This proposal moves the warning from the call site to the method declaration.

    static List<List<String>> listOfListOfStrings() {
      List<String> a = new ArrayList<String>(), b = new ArrayList<String>(), c = new ArrayList<String>();

      // Warning: [unchecked] unchecked generic array creation for varargs parameter of type List<String>[]
      return Arrays.asList(a, b, c);
    }

It looks like a new annotation is being planned to support this at the declaration and varargs warnings will be revised. Callers may be able to remove @SuppressWarnings(“unchecked”) from their code once this is supported by the compiler. This support is not yet available as of build b114.

Improved Type Inference for Generic Instance Creation (Diamond)

This one is much easier of all the Coin proposals, at least in terms of simplicity and eliminates much of the Generics verbosity. Generics enforced type redundancy during declaration. For example, the traditional way (>=1.5) of declaring a Generic type is shown below.

    static class Library<E> {
        Set<Method> methods = new HashSet<Method>();

        Library(Set<Method> methods) {
            this.methods = methods;
        }
    }

    Set<Method> methods = new HashSet<Method>();
    Library<Set<Method>> library = new Library<Set<Method>>(methods);

Now, with this language support, the declaration can be simplified as shown below:

    static class Library<E> {
        Set<Method> methods = new HashSet<>(); // saves typing

        Library(Set<Method> methods) {
            this.methods = methods;
        }
    }

    Set<Method> methods = new HashSet<>(); // saves typing
    Library<Set<Method>> library = new Library<>(methods); // saves typing

While libraries such as Guava have attempted similar support for this, language level sugar is more intuitive than alternatives. This feature was implemented more than a year ago.

Improved Exception Handling for Java

Prior to Java 7, it was very cumbersome to catch multiple exception types leading to boilerplate code. This proposal addresses this concern by catching multiple exception types with a single catch clause and improves checking for rethrown exceptions.

Here is a simple class we will be using to test reflection:

public class CoinByReflection {
    public int code = 10_10_2010;

    public void printTheme() {
        System.out.println("Making things programmers do everyday easier.");
    }
}

This pattern is noticed when working with reflective operations where we may need to handle different types of exceptions.

    public void multicatch() throws ClassNotFoundException, InstantiationException, NoSuchMethodException, NoSuchFieldException, IllegalAccessException, InvocationTargetException {
        try {
            Class clazz = Class.forName("CoinByReflection");
            Object instance = clazz.newInstance();
            Field field = clazz.getField("code");
            int code = field.getInt(instance);
            System.out.println(code);
            Method method = clazz.getMethod("printTheme");
            method.invoke(instance);
        } catch (ClassNotFoundException cnfe) {
            log(cnfe);
            throw cnfe;
        } catch (InstantiationException ie) {
            log(ie);
            throw ie;
        } catch (NoSuchMethodException nsme) {
            log(nsme);
            throw nsme;
        } catch (NoSuchFieldException nsfe) {
            log(nsfe);
            throw nsfe;
        } catch (IllegalAccessException iae) {
            log(iae);
            throw iae;
        } catch (InvocationTargetException ite) {
            log(ite);
            throw ite;
        }
    }

With this proposal, we should be able to simplify the exception handling when performing reflection with single catch.

    public void singlecatch() throws ClassNotFoundException, InstantiationException, NoSuchMethodException, NoSuchFieldException, IllegalAccessException, InvocationTargetException {
        try {
            Class clazz = Class.forName("CoinByReflection");
            Object instance = clazz.newInstance();
            Field field = clazz.getField("code");
            int code = field.getInt(instance);
            System.out.println(code);
            Method method = clazz.getMethod("printTheme");
            method.invoke(instance);
        } catch (final ClassNotFoundException|
        InstantiationException |
                NoSuchMethodException |
                NoSuchFieldException |
                IllegalAccessException |
                InvocationTargetException
        e){
            log(e);
            throw e;
        }
    }

ReflectiveOperationException, a new common super class of exceptions thrown by reflective operations was introduced to simplify this further. You can catch this exception and precisely rethrow one of the exceptions defined in the method signature.

    public void singlecatch() throws ClassNotFoundException, InstantiationException, NoSuchMethodException, NoSuchFieldException, IllegalAccessException, InvocationTargetException {
        try {
            Class clazz = Class.forName("CoinByReflection");
            Object instance = clazz.newInstance();
            Field field = clazz.getField("code2");
            int code = field.getInt(instance);
            System.out.println(code);
            Method method = clazz.getMethod("printTheme");
            method.invoke(instance);
        } catch (final ReflectiveOperationException e){
            log(e);
            throw e;
        }
    }

This feature was implemented in May.

Conclusion

With more than 80% of the Coin features implemented and “ready” for battle test, it is evident from the fact that these features live up to its expectations:

Making things programmers do everyday easier.

[as quoted from JavaOne Coin presentation]. Now that we have a definitive plan for Java 7, IDEs should start supporting these features as mid-2011 is not very far and play a vital role in accelerating adoption of these new syntaxes. It looks like Coin 2 will include other interesting proposals such as Collection literals, Multi-line strings, Large arrays, etc, possibly for Java 8?

Recently, NetBeans announced that it will support JDK 7 language features in its forthcoming 7.0 release and Eclipse plans to include Java 7 support in its next release 3.7 (Indigo). Intellij 9 has some support for Java 7, and I hope Intellij X brings the rest.

Possibly Related Posts:


In Flux, we have made improvements in recent versions to handle database or network failures and recover from such failures gracefully without needing to restart Flux. Sometimes, it would be nice to notify administrators about these failures and they can act upon them in case of an unscheduled outage. Flux is dependent on the database for maintaining job states and schedules, so fixing this problem is critical.

BoneCP, a popular JDBC Connection Pool has an interesting and useful feature which allows the connection pool to automatically recover from such DB outages and it replays the transactions when a healthy connection becomes available. This feature became available since its 0.6.5 release and I had tested this before and found this to be very useful when configured with Flux.

In BoneCP, we can implement a connection hook which gets triggered when database failures occur and it is easy to configure in Flux to use BoneCP as the datasource pool provider. In case of any database failure, you may want to send an email notification or initiate a SNMP trap so other downstream systems can be handled accordingly.

Here is a basic DatabaseShutdownHook would look like:

import com.jolbox.bonecp.ConnectionHandle;
import com.jolbox.bonecp.hooks.AbstractConnectionHook;
import com.jolbox.bonecp.hooks.AcquireFailConfig;

import java.util.Date;

public class DatabaseShutdownHook extends AbstractConnectionHook {
    @Override
    public boolean onConnectionException(ConnectionHandle connection, String state, Throwable t) {
        // handle notifications here: SNMP or SMTP
        System.out.println("Database down at " + new Date());
        return super.onConnectionException(connection, state, t);
    }

    @Override
    public boolean onAcquireFail(Throwable t, AcquireFailConfig acquireConfig) {
        // handle notifications here: SNMP or SMTP
        System.out.println("Failure to acquire connection at " + new Date() + ". Retry attempts remaining : " + acquireConfig.getAcquireRetryAttempts());
        return super.onAcquireFail(t, acquireConfig);
    }

}

Let us now see how to configure BoneCP as a Data source in Flux.

import com.jolbox.bonecp.BoneCPDataSource;
import flux.Configuration;
import flux.DatabaseType;
import flux.Engine;
import flux.Factory;

import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;
import java.sql.Connection;
import java.sql.SQLException;

public class FluxEngine {

    private static Context initialContext;

    static {
        try {
            System.setProperty(Context.INITIAL_CONTEXT_FACTORY, "org.apache.naming.java.javaURLContextFactory");
            System.setProperty(Context.URL_PKG_PREFIXES, "org.apache.naming");
            initialContext = new InitialContext();
            initializeDataSource();
        } catch (NamingException e) {
            // log exception
        } catch (SQLException e) {
            // log exception
        }
    }

    public static void initializeDataSource()  throws NamingException, SQLException {
        BoneCPDataSource ds = new BoneCPDataSource();
        ds.setJdbcUrl("jdbc:mysql://localhost:3306/flux710?relaxAutoCommit=true");
        ds.setUsername("flux");
        ds.setPassword("secret");
        ds.setMinConnectionsPerPartition(10);
        ds.setMaxConnectionsPerPartition(50);
        ds.setPartitionCount(1);
        ds.setConnectionHook(new DatabaseShutdownHook());// Required only if you need notifications.
        ds.setTransactionRecoveryEnabled(true);// Important: This should be enabled
        ds.setAcquireRetryAttempts(10);//default is 5
        ds.setAcquireRetryDelay(10000);// default is 7 secs
        ds.setReleaseHelperThreads(5);

        Connection con = ds.getConnection();
        if (con != null) {
            initialContext.rebind("FluxDataSource", ds);
            con.close();
        }
        System.out.println("DataSource configured.");
    }

    public static void main(String[] args) throws Exception {
        Factory f = Factory.makeInstance();
        Configuration c = f.makeConfiguration();
        c.setDatabaseType(DatabaseType.MYSQL);
        c.setDataSource("FluxDataSource");
        Engine engine = f.makeEngine(c);
        engine.start();
        System.out.println("Engine started.");
    }
}

Data sources should be made available via JNDI for Flux to use them. So, in this example I used Tomcat JNDI support to expose the BoneCP datasource via JNDI. There are also other ways to expose this via JNDI when running Flux as a standalone server. Tomcat JNDI was easy to configure though as you can see from the code above. The BoneCP data source should be configured for transaction recovery and you can set the number of recovery attempts and retry delay.

I have created a simple flow chart which has a Timer Trigger followed by a Java Action. The timer is configured to fire every 15 seconds for 5 times as shown below.

process_data.png

Here is a sample output of running this job in Flux configured with BoneCP and MySQL database. I shutdown MySQL server when this job was running and you can see the BoneCP recovery attempts and after a while I brought back the MySQL server and BoneCP successfully recovered. Now, you can see Flux execute the last occurrence of this job successfully.

DataSource configured.
Engine started.
Done processing data

Done processing data

Done processing data

Done processing data

Database down at Sun Oct 10 12:18:12 MDT 2010
Oct 10, 2010 12:18:12 PM com.jolbox.bonecp.ConnectionHandle markPossiblyBroken
SEVERE: Database access problem. Killing off all remaining connections in the connection pool. SQL State = 08007
Oct 10, 2010 12:18:12 PM com.jolbox.bonecp.MemorizeTransactionProxy invoke
SEVERE: Connection failed. Attempting to recover transaction on Thread #70
Oct 10, 2010 12:18:14 PM com.jolbox.bonecp.hooks.AbstractConnectionHook onAcquireFail
SEVERE: Failed to acquire connection Sleeping for 10000ms and trying again. Attempts left: 10. Exception: java.net.ConnectException: Connection refused: connect
Failure to acquire connection at Sun Oct 10 12:18:14 MDT 2010. Retry attempts remaining : 10
Oct 10, 2010 12:18:26 PM com.jolbox.bonecp.hooks.AbstractConnectionHook onAcquireFail
SEVERE: Failed to acquire connection Sleeping for 10000ms and trying again. Attempts left: 9. Exception: java.net.ConnectException: Connection refused: connect
Failure to acquire connection at Sun Oct 10 12:18:26 MDT 2010. Retry attempts remaining : 9
Oct 10, 2010 12:18:38 PM com.jolbox.bonecp.hooks.AbstractConnectionHook onAcquireFail
SEVERE: Failed to acquire connection Sleeping for 10000ms and trying again. Attempts left: 8. Exception: java.net.ConnectException: Connection refused: connect
Failure to acquire connection at Sun Oct 10 12:18:38 MDT 2010. Retry attempts remaining : 8
Oct 10, 2010 12:18:48 PM com.jolbox.bonecp.MemorizeTransactionProxy invoke
SEVERE: Recovery succeeded on Thread #70
Done processing data

The following BoneCP dependencies are required in the classpath:
bonecp-0.7.0.jar
guava-r07.jar
slf4j-api-1.6.1.jar
slf4j-jdk14-1.6.1.jar (Note: You can use any logger bindings supported by slf4j. Using JDK logger for simplicity and one less jar.)

To configure Tomcat JNDI, you need to have these jars in classpath:
catalina.jar
tomcat-juli.jar

Let me know if you have any trouble setting up this in Flux.

(Update: 10/13): Updated to reference the latest stable 0.7.0 release which now uses Google guava instead of the retired Google collections library and fixed slf4j dependency requirement.

Possibly Related Posts: