Yahoo Search provides a RESTful interface for web search. Of course this service is been there for a while. I was just curious to use Jersey Client to play around with the Yahoo REST service. It opens up a world of possibilities for building applications around these services in Java.

Let us invoke the web search API using GET and context search API using POST as described in the following URLs.

http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=finances&format=pdf (GET)

http://search.yahooapis.com/WebSearchService/V1/contextSearch (POST)
appid=YahooDemo&query=madonna&context=Italian+sculptors+and+painters+of+the+renaissance+favored+the+Virgin+Mary+for+inspiration

The URL is constructed using the actual query parameters for accessing web resources which takes the URL encoded “argument=value” form.

The below code shows how we can quickly write a Jersey based REST client for invoking these RESTful web services. The APIs are fairly self-explanatory.

import com.sun.jersey.api.client.Client;
import com.sun.jersey.api.client.WebResource;
import com.sun.jersey.api.representation.Form;
import restful.search.jaxb.ResultSet;
import restful.search.jaxb.ResultType;

import javax.ws.rs.core.MediaType;

public class YahooRestfulSearchClient {
    public static void main(String args[]) {
        Client client = Client.create();
        WebResource r = client.resource("http://search.yahooapis.com/WebSearchService/V1/webSearch?appid=YahooDemo&query=finances&format=pdf");
        ResultSet searchResults = r.accept(MediaType.APPLICATION_XML).get(ResultSet.class);
        for (ResultType result : searchResults.getResult()) {
            System.out.println("Title : " + result.getTitle());
            System.out.println("Summary : " + result.getSummary());
            System.out.println("Url : " + result.getUrl());
        }

        Form form = new Form();
        form.add("appid", "YahooDemo");
        form.add("query", "madonna");
        form.add("context", "Italian sculptors and painters of the renaissance favored the Virgin Mary for inspiration");

        r = client.resource("http://search.yahooapis.com/WebSearchService/V1/contextSearch");
        searchResults = r.accept(MediaType.APPLICATION_XML).post(ResultSet.class, form);        
        for (ResultType result : searchResults.getResult()) {
            System.out.println("Title : " + result.getTitle());
            System.out.println("Summary : " + result.getSummary());
            System.out.println("Url : " + result.getUrl());
        }        
    }
}

I updated my original approach for invoking the Context search based on the suggestions from Jersey developers to use Jersey form parameters encoded in the body of the request instead of directly constructing the URI as a request entity in the post call. Typically, POST is used to create web resources. But, Yahoo documentation suggests that POST be used for context searches just because the query params can sometimes be potentially lengthy, and hence the recommendation to send query parameters in the POST data block. The other option in Jersey is to use the UriBuilder to construct the URI and add query parameters to the URI. The code snip below shows this latter approach. Using form parameters is more elegant than constructing lengthy URIs 🙂


        URI uri = UriBuilder.fromPath("http://search.yahooapis.com/WebSearchService/V1/contextSearch").queryParam("appid", "YahooDemo").queryParam("query", "madonna").queryParam("context", "Italian sculptors and painters of the renaissance favored the Virgin Mary for inspiration").build();
        r = client.resource(uri);
        searchResults = r.accept(MediaType.APPLICATION_XML).post(ResultSet.class);

In both these approaches, the crux lies in the response type ‘ResultSet’ which is described in the below schema for the responses returned from the Yahoo REST service. You can access the schema here. Here is a quick look at the schema.

<?xml version="1.0" encoding="utf-8" ?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema"
  targetNamespace="urn:yahoo:srch"
  xmlns="urn:yahoo:srch"
  elementFormDefault="qualified">

	<xs:element name="ResultSet">
		<xs:complexType>
			<xs:sequence>
				<xs:element name="Result" type="ResultType" minOccurs="0" maxOccurs="100" />
			</xs:sequence>
			<xs:attribute name="totalResultsAvailable" type="xs:integer" />
			<xs:attribute name="totalResultsReturned" type="xs:integer" />

			<xs:attribute name="firstResultPosition" type="xs:integer" />
		</xs:complexType>
	</xs:element>

	<xs:complexType name="ResultType">
		<xs:sequence>
			<xs:element name="Title" type="xs:string" />
			<xs:element name="Summary" type="xs:string" />
			<xs:element name="Url" type="xs:string" />

			<xs:element name="ClickUrl" type="xs:string" />
			<xs:element name="ModificationDate" type="xs:string" minOccurs="0" />
			<xs:element name="MimeType" type="xs:string" minOccurs="0" />
			<xs:element name="Cache" type="CacheType" minOccurs="0" />
		</xs:sequence>
	</xs:complexType>

	<xs:complexType name="CacheType">
		<xs:sequence>
			<xs:element name="Url" type="xs:string" />

			<xs:element name="Size" type="xs:string" />
		</xs:sequence>
	</xs:complexType>

</xs:schema>

Use this schema to generate your JAXB types (ResultSet, ResultType, CacheType) and you are ready to consume the response within your REST client. The best part of this exercise was using IntelliJ IDEA 8.0M1. This latest release has support for REST client plugin where you can test your restful resources and it supports RESTful development using Jersey. More to explore on this front. Here is a screen shot of IDEA8 Rest Client in action.
IDEA8 Rest Client

Possibly Related Posts:


4 Responses to “Yahoo Search Restful Client using Jersey”

  • andjarnic

    I am curious why yahoo supports a POST call to do a search…?? Usually post is used to create something, yet according to there api page, its just like get with the use of contextSearch. Are they using post to differentiate a regular search from a context search? Seems like a query parameter would work fine in the GET version.

  • admin

    Andjarnic,

    Yahoo documentation suggests that POST be used for context searches just because the URL can sometimes be potentially lengthy due to the large number of context parameters supported. Query parameters with GET works just fine for context searches too. But, this is not a recommendation. I just updated my blog entry to use form parameters as well as query parameters supported by Jersey Client API for performing context searches using POST.

    -Arul

Trackbacks/Pingbacks

  1.  Recent Links Tagged With "restful" - JabberTags
  2.  Un cliente Jersey que consume Google AJAX search API – Rest & Java « Programa en Java y disfruta de un café lojano