Tuesday, August 11, 2009

JavaScript Regex to Format a Monetary Amount

Found this lying around in my old notes, it's a javascript regular expression that will format a monetary string (example: $100,000.00) into a straight numeric value:

replace(/[^0-9//-//.]/g, "");


The expression will strip out any of the non-numeric characters not allowed in normal decimal arithmetic.

Example of use:

alert("$100,000.00".replace(/[^0-9//-//.]/g, ""));

produces an alert box displaying 100000.00

Thursday, August 6, 2009

Building a Building a Custom Trust Association Interceptor for WebSphere Portal, Part IV

So far, in Parts I, II, and III we've seen how to

  • Create a custom TAI for WebSphere Portal
  • Install the TAI on the server
  • Create a simple PGP security class

In the final part of this series, we'll examine how to use the PGP class in our custom TAI.

Updated code:



package test.security.tai;

import java.rmi.RemoteException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Properties;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import com.ibm.websphere.security.CustomRegistryException;
import com.ibm.websphere.security.EntryNotFoundException;
import com.ibm.websphere.security.UserRegistry;
import com.ibm.websphere.security.WebTrustAssociationException;
import com.ibm.websphere.security.WebTrustAssociationFailedException;
import com.ibm.wsspi.security.tai.TAIResult;
import com.ibm.wsspi.security.tai.TrustAssociationInterceptor;
import com.security.pgp.PGPSSOUtil

/**
* Custom Login Module
*
* Project imports the jar wssec.jar for development purposes.
* Found in the server runtime lib directory ($irad_home$\runtimes\base_v6\)
*
*
**/
public class CustomPortalTAI implements TrustAssociationInterceptor
{
private static final String VERSION = "Custom TAI version 1.0 \n Author: SirCrofty \n " + "Last Updated: March 1, 2008";
private static final String TYPE = "--- Custom TAI --- \n Custom Trust Assocation Interceptor for WebSphere Portal Application";
HashMap sharedState = null;

/**
* Constructor
*
**/
public CustomPortalTAI()
{
sharedState = new HashMap();
}

/**
* (non-Javadoc)
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#initialize(java.util.Properties)
* @param arg0
* @return
* @throws com.ibm.websphere.security.WebTrustAssociationFailedException
*
**/
public int initialize(Properties props) throws WebTrustAssociationFailedException
{
return 0;
}


/**
* (non-Javadoc)
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#isTargetInterceptor(javax.servlet.http.HttpServletRequest)
* @param arg0
* @return
* @throws com.ibm.websphere.security.WebTrustAssociationException
*
**/
public boolean isTargetInterceptor(HttpServletRequest req) throws WebTrustAssociationException
{
System.out.println("*********** Custom TAI ******************");
System.out.println("Determining if this TAI should handle the incoming request...");

if (req.getParameter("customUser") != null)
{
System.out.println("Custom TAI is being used to establish trust!");
return true;
}

System.out.println("Bypassing Custom TAI, did not find a user ID in the request");
return false;
}

/**
* (non-Javadoc)
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#negotiateValidateandEstablishTrust(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse)
* @param arg0
* @param arg1
* @return
* @throws com.ibm.websphere.security.WebTrustAssociationFailedException
*
**/
public TAIResult negotiateValidateandEstablishTrust(HttpServletRequest req, HttpServletResponse resp)
throws WebTrustAssociationFailedException
{
String encryptId = req.getParameter("customUser");
try
{
PGPSSOUtil util = new PGPSSOUtil();
// assume the user id has been encrypted and then converted to Hex encoding
byte[] unHexBytes = util.convertHexStringToByteArray(encryptId);
// use the keys referenced in PGPSSOUtil to decrypt the userId
userId = util.decryptId(unHexBytes);
}
catch (Exception e)
{
return TAIResult.create(HttpServletResponse.SC_FORBIDDEN, userId);
}

return TAIResult.create(HttpServletResponse.SC_OK, userId);
}



/**
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#cleanup()
*
*
**/
public void cleanup()
{
sharedState = null;
}


/**
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#getType()
* @return
*
**/
public String getType()
{
return TYPE + " \n " + this.getClass().getName();
}

/**
*
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#getVersion()
* @return
*
**/
public String getVersion()
{
return VERSION;
}

}


The parts of interest here are the changes to the negotiateValidateandEstablishTrust method.

We've replaced the conditional that checks the user name value against a constant to instead use the encrypted user id passed in the request. Assuming the id decrypts correctly (userId = util.decryptId(unHexBytes);), we'll pass the userId along to the Portal itself for authorization. If the user name does not encrypt correctly, than the decryption will fail, an exception will be thrown, and a Forbidden access message will be displayed. In this way, we use the security and access controls already built into WebSphere Portal to handle the rest of the job once we have an acceptable user id.

The only other thing to note is the decoding of the hashed user id from Hex. This is done to because we are assuming the user id has been hex encoded so that it can be passed around while avoiding a bunch of special characters.

This series should hopefully give you a jump on how to create a simple SSO solution for WebSphere Portal. For production purposes, there would be other considerations, such as the storage of the PGP key files and how to determine the location of the trusted sender, but this should get you started in the right direction.

Monday, July 27, 2009

Building a Building a Custom Trust Association Interceptor for WebSphere Portal, Part III

Continuing the discussion on a quick and easy way of accomplishing single sign on to WebSphere Portal (see Part 1 and Part 2), we look at how to decrypt incoming parameters.

There is obviously a very easy to exploit security issue if authentication parameters are passed between different applications in clear text. Any user with a HTTP proxy (such as Charles) would be able to listen on requests and determine authentication protocol. The following solution allows the trusted third party to encrypt authentication information being passed between the portal site and the third party.

For the purpose of this exercise, we'll use PGP for decryption. PGP is available for encryption/decryption, but it is no longer distributed as freeware. You need to download PGP Corp's Desktop Trial Software to aquire this functionality now. Don't worry - the software reverts to the freeware functionality after 30 days, so you'll still be able to do basic encryption and decryption.

This examples assumes you already have gone through the process of generating a private key file and a public key file, and placed those somewhere on your server. The code later on will need to reference those key ring files. If you need help creating a key ring, see the PGP documentation for help.

We'll use the Bouncy Castle API to perform the nitty-gritty process of decrypting the incoming information. You'll need to download the appropriate version of the bcprov-jdk*.jar and bcpg-jdk*.jar from the Latest Releases page. Add these files to the lib directory of the application server, so that they are available to the server itself.

Now, the catch to using PGP is that it is a strong encryption algorithm. So strong, in fact, that you need to download a different java security policy to work with it. For the purpose of WebSphere Portal, you would get them off the IBM site (here). For different jdk providers, you would get it from their respective sites (such as Sun, Oracle). Unpack these files into the jre/lib/security directory of your jdk.

Great. Now that we're done with the necessary set up and dependencies, let's look at the code. A lot of this can be found as an example in the Bouncy Castle documentation. Remember to change the constants at the beginning of the class to match your secret pass phrase and the location of your key ring files.



/*
* PGP Decryption Utility
*
*/
package com.security.pgp;
import java.io.BufferedInputStream;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.security.NoSuchProviderException;
import java.security.Security;
import java.util.Iterator;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.openpgp.PGPCompressedData;
import org.bouncycastle.openpgp.PGPEncryptedDataList;
import org.bouncycastle.openpgp.PGPException;
import org.bouncycastle.openpgp.PGPLiteralData;
import org.bouncycastle.openpgp.PGPObjectFactory;
import org.bouncycastle.openpgp.PGPPrivateKey;
import org.bouncycastle.openpgp.PGPPublicKey;
import org.bouncycastle.openpgp.PGPPublicKeyEncryptedData;
import org.bouncycastle.openpgp.PGPPublicKeyRing;
import org.bouncycastle.openpgp.PGPPublicKeyRingCollection;
import org.bouncycastle.openpgp.PGPSecretKey;
import org.bouncycastle.openpgp.PGPSecretKeyRingCollection;
import org.bouncycastle.openpgp.PGPUtil;

public class PGPSSOUtil
{
private static final String PROVIDER_SHORT = "BC";
private static final String KEY_PHRASE = "MySecretPhrase123";
private static final String DECRYPTION_KEY_FILE = "C:\\PGP\\PGP60.pkr";
private static final String PRIVATE_KEY_FILE = "C:\\PGP\\PGP60.skr";

private static PGPPublicKey pubKey;
private static PGPPrivateKey privateKey;

static
{
// adds the BouncyCastle Security Provider to the Java Security Providers
// list
Security.addProvider(new BouncyCastleProvider());
}

/**
* Constructor
*
* Initializes PGPPublicKey and PGPPrivateKey objects
*
* @throws Exception
*
**/
public PGPSSOUtil() throws Exception
{
// get the PGPPublicKey object
if (pubKey == null)
{
FileInputStream fis = new FileInputStream(DECRYPTION_KEY_FILE);
pubKey = readPublicKeyFromCol(fis);
fis.close();
}
// get the PGPPrivateKey object
if (privateKey == null)
{
FileInputStream fis = new FileInputStream(PRIVATE_KEY_FILE);
privateKey = findSecretKey(fis, pubKey.getKeyID(), KEY_PHRASE.toCharArray());
fis.close();
}
}

/**
* Given an encypted hex byte array, decrypt the data and return a user id to the caller
*
* @param encryptHex Byte Array, containing the encypted user id
* @return a valid SW1 user id, or whatever else is contained in the ecrypted data
*
**/
public String decryptId(byte[] encryptHex)
{
String id = null;
if (encryptHex == null encryptHex.length == 0)
return id;

try
{
// get the decrypted data
byte[] decryptedData = decrypt(privateKey, encryptHex);
// return a String translation of the Byte Array
if (decryptedData != null)
id = new String(decryptedData);
}
catch (Exception e)
{
e.printStackTrace();
}

return id;
}


/**
* Decrypt the given data using the BouncyCastle PGP provider
*
* @param privKey PGPPrivateKey object to decrypt the data with
* @param input Byte Array to decrypt
* @return Byte Array of decypted data
* @throws Exception
*
**/
private byte[] decrypt(PGPPrivateKey privKey, byte[] input) throws Exception
{
byte[] decrypted = null;
try
{
int bufferSize = 1024;
InputStream in = new ByteArrayInputStream(input);
in = PGPUtil.getDecoderStream(in);
PGPObjectFactory pgpObjectFactory = new PGPObjectFactory(in);
PGPEncryptedDataList enc;
Object pgpObject = pgpObjectFactory.nextObject();
if (pgpObject instanceof PGPEncryptedDataList)
{
enc = (PGPEncryptedDataList) pgpObject;
}
else
{
enc = (PGPEncryptedDataList) pgpObjectFactory.nextObject();
}
Iterator it = enc.getEncryptedDataObjects();
PGPPublicKeyEncryptedData pbe = null;
while (it.hasNext())
{
pbe = (PGPPublicKeyEncryptedData) it.next();
}
if (pbe == null)
{
return decrypted;
}
InputStream clear = pbe.getDataStream(privateKey, PROVIDER_SHORT);
PGPObjectFactory plainFact = new PGPObjectFactory(clear);
PGPCompressedData cData = (PGPCompressedData)
plainFact.nextObject();
InputStream compressedStream = cData.getDataStream();
if (!(compressedStream instanceof BufferedInputStream))
{
compressedStream = new BufferedInputStream(compressedStream, bufferSize);
}
PGPObjectFactory pgpFact = new PGPObjectFactory(compressedStream);
Object message = pgpFact.nextObject();
if (message instanceof PGPLiteralData)
{
PGPLiteralData literalData = (PGPLiteralData) message;
ByteArrayOutputStream fOut = new ByteArrayOutputStream();
BufferedInputStream unc = new
BufferedInputStream(literalData.getInputStream(), bufferSize);
byte[] buffer = new byte[bufferSize];
int bytesRead;
while ((bytesRead = unc.read(buffer)) != -1)
{
fOut.write(buffer, 0, bytesRead);
fOut.flush();
}
decrypted = fOut.toByteArray();
fOut.close();
in.close();
}
}
catch (Exception e)
{
e.printStackTrace();
throw e;
}
return decrypted;
}


/**
* Find the public key using the BouncyCastle Provider.
*
* @param in InputStream A InputStream created from the PGP Public Key file
* @return PGPPublicKey object used for encrypting and decypting data
* @throws Exception
*
**/
private PGPPublicKey readPublicKeyFromCol(InputStream in) throws Exception
{
PGPPublicKeyRing pkRing = null;
// get the Public Key Rings from the given Public Key File
PGPPublicKeyRingCollection pkCol = new PGPPublicKeyRingCollection(in);
Iterator it = pkCol.getKeyRings();
while (it.hasNext()) // loop over the key rings
{
pkRing = (PGPPublicKeyRing) it.next();
Iterator pkIt = pkRing.getPublicKeys();
while (pkIt.hasNext())
{
PGPPublicKey key = (PGPPublicKey) pkIt.next();
// check if it's the encryption key, this is the one we want to decrypt files with
if (key.isEncryptionKey())
{
return key;
}
}
}
return null;
}


/**
* Find the PGP Private Key from the Private Key file, using BouncyCastle
*
* @param keyIn InputStream created from the PrivateKey File location
* @param keyID The Key Id from the Public Key File
* @param pass The Passphrase used to extract the private key
* @return PGPPrivateKey representing the private key to decrypt with
* @throws IOException
* @throws PGPException
* @throws NoSuchProviderException
*
**/
private static PGPPrivateKey findSecretKey(InputStream keyIn, long keyID, char[] pass) throws IOException, PGPException, NoSuchProviderException
{
PGPSecretKeyRingCollection pgpSec = new PGPSecretKeyRingCollection(PGPUtil.getDecoderStream(keyIn));
PGPSecretKey pgpSecKey = pgpSec.getSecretKey(keyID);
if (pgpSecKey == null)
{
return null;
}
return pgpSecKey.extractPrivateKey(pass, PROVIDER_SHORT);
}


/**
* Converts a Hex String back to a Byte Array. Courtesy Bouncy Castle
*
* @param hexStr Hex String to convert back to a normal byte array
* @return normally encoded byte array
*
**/
public byte[] convertHexStringToByteArray(String hexStr)
{
byte bArray[] = new byte[hexStr.length()/2];
for(int i=0; i<(hexStr.length()/2); i++)
{
byte firstNibble = Byte.parseByte(hexStr.substring(2*i,2*i+1),16);
byte secondNibble = Byte.parseByte(hexStr.substring(2*i+1,2*i+2),16);
int finalByte = (secondNibble) | (firstNibble << 4 );
bArray[i] = (byte) finalByte;
}
return bArray;
}

/**
* Converts a Byte Array to Hex String. Got this online from Bouncy Castle
*
* @param in byte[] array normally encoded
* @return String in Hex format
*
**/
public String convertByteArrayToHexString(byte in[])
{
byte ch = 0x00;
int i = 0;
if (in == null || in.length <= 0)
{
return null;
}
String pseudo[] = {"0", "1", "2", "3", "4", "5", "6", "7", "8","9", "A", "B", "C", "D", "E", "F"};
StringBuffer out = new StringBuffer(in.length * 2);
while (i < in.length)
{
ch = (byte) (in[i] & 0xF0);
ch = (byte) (ch >>> 4);
ch = (byte) (ch & 0x0F);
out.append(pseudo[ (int) ch]);
ch = (byte) (in[i] & 0x0F);
out.append(pseudo[ (int) ch]);
i++;
}
String rslt = new String(out);
return rslt;
}


}


Now we would import this class into the original class developed in Part I and use it in
the negotiateValidateandEstablishTrust method. See Part IV for the changes to the class we originally developed.

Wednesday, May 6, 2009

Turning a Wii controller into a Smart Board

Found a fantastic video on YouTube about how to use a Wiimote to turn any suface into an interactive white board.

http://www.youtube.com/watch?v=5s5EvhHy7eQ

This is definitely pretty cool.

Thursday, April 16, 2009

Building a Custom Trust Association Interceptor for WebSphere Portal, Part II

In Part I, we looked at the code for building a simple SSO Trust Association Interceptor for WebSphere Portal. This part explains the general steps on how to install the TAI on the application server.

The following steps describe how to Install a custom Trust Association Interceptor for WebSphere.

1.) Develop a class that extends TrustAssociationInterceptor
  • Fully qualified class for WebSphere is com.ibm.wsspi.security.tai.TrustAssociationInterceptor
  • Need to override the following methods:
  • initialize: initializes the TAI
  • isTargetInterceptor: determines if this TAI should be used as the one to check for Trust Association for the requested resource
  • negotiateValidateandEstablishTrust: this method does the actual checking to see if we can create the custom TAI Subject that is passed to the underlying protected resource
2.) Add the following libraries to the build path - (for development purposes. Project will have compilation errors without them)
  • sas.jar
  • wssec.jar
3.) Export the jar
  • Needs to be installed in the /lib/ext directory on all nodes for the application server
  • Any other necessary jar files used by the TAI should be placed in the /lib directory
4.) Configure the TAI on the ND
  • Security > Authentication Mechanisms > LPTA > Trust Association > Interceptors
  • Choose to create a new Interceptor
  • Enter the fully qualified class name of the Interceptor class (package + class name)
  • Apply, then Ok
6.) Enable Security on both WebSphere Application server and WebSphere Portal (if this hasn't been done yet). See here for more information.

5.) Restart all nodes

You should be able to see a print out at Server start up indicating it has loaded the new TAI.

Building a Custom Trust Association Interceptor for WebSphere Portal

At one point, I had to develop a custom Single Sign On solution to WebSphere Portal. The general standard for this is to use TAM, but since this wasn't available, I wrote something a little more simple, using a Trust Association Interceptor. Trust Association Interceptors are used when a page in a Web Application has been marked as protected. The TAI can determine if a user should be allowed to access that page.

This multiple part posting explores setting up a simple TAI for WebSphere Portal. It's a collection of information around the web. For an excellent resource on Trust Association Interceptors, see this article (which I wish I had seen when I did this).

As usual, the code has been stripped down to the key parts.

Part I: The code

package test.security.tai;

import java.rmi.RemoteException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.Iterator;
import java.util.List;
import java.util.Properties;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import com.ibm.websphere.security.CustomRegistryException;
import com.ibm.websphere.security.EntryNotFoundException;
import com.ibm.websphere.security.UserRegistry;
import com.ibm.websphere.security.WebTrustAssociationException;
import com.ibm.websphere.security.WebTrustAssociationFailedException;
import com.ibm.wsspi.security.tai.TAIResult;
import com.ibm.wsspi.security.tai.TrustAssociationInterceptor;

/**
* Custom Login Module
*
* Project imports the jar wssec.jar for development purposes.
* Found in the server runtime lib directory ($irad_home$\runtimes\base_v6\)
*
*
**/
public class CustomPortalTAI implements TrustAssociationInterceptor
{
private static final String VERSION = "Custom TAI version 1.0 \n Author: SirCrofty \n " + "Last Updated: March 1, 2008";

private static final String TYPE = "--- Custom TAI --- \n Custom Trust Assocation Interceptor for WebSphere Portal Application";

HashMap sharedState = null;

/**
* Constructor
*
**/
public CustomPortalTAI()
{
sharedState = new HashMap();
}

/**
* (non-Javadoc)
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#initialize(java.util.Properties)
* @param arg0
* @return
* @throws com.ibm.websphere.security.WebTrustAssociationFailedException
*
**/
public int initialize(Properties props) throws WebTrustAssociationFailedException
{
return 0;
}


/**
* (non-Javadoc)
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#isTargetInterceptor(javax.servlet.http.HttpServletRequest)
* @param arg0
* @return
* @throws com.ibm.websphere.security.WebTrustAssociationException
*
**/
public boolean isTargetInterceptor(HttpServletRequest req) throws WebTrustAssociationException
{
System.out.println("*********** Custom TAI ******************");
System.out.println("Determining if this TAI should handle the incoming request...");

if (req.getParameter("customUser") != null)
{
System.out.println("Custom TAI is being used to establish trust!");
return true;
}


System.out.println("Bypassing Custom TAI, did not find a user ID in the request");
return false;
}
/**
* (non-Javadoc)
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#negotiateValidateandEstablishTrust(javax.servlet.http.HttpServletRequest, javax.servlet.http.HttpServletResponse)
* @param arg0
* @param arg1
* @return
* @throws com.ibm.websphere.security.WebTrustAssociationFailedException
*
**/
public TAIResult negotiateValidateandEstablishTrust(HttpServletRequest req, HttpServletResponse resp)
throws WebTrustAssociationFailedException
{
String userId = req.getParameter("customUser");
if (userId.equals("portalUser"))
{
System.out.println("*********** CustomTAI *****************");
System.out.println("UserID = " + userId);

return TAIResult.create(HttpServletResponse.SC_OK, userId);
}
else
{
return TAIResult.create(HttpServletResponse.SC_FORBIDDEN, userId);
}
}

/**
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#cleanup()
*
*
**/
public void cleanup()
{
sharedState = null;
}


/**
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#getType()
* @return
*
**/
public String getType()
{
return TYPE + " \n " + this.getClass().getName();
}

/**
*
* @see com.ibm.wsspi.security.tai.TrustAssociationInterceptor#getVersion()
* @return
*
**/
public String getVersion()
{
return VERSION;
}
}

We're implementing the com.ibm.wsspi.security.tai.TrustAssociationInterceptor interface provided by IBM. In order to get this to compile, you may need to place some jars in your class path during development. These jars are named wssec.jar and sas.jar, and are found in your server runtime directory.

The two methods of interest above are isTargetInterceptor and negotiateValidateandEstablishTrust. Both of these methods accept an HttpServletRequest as input, and that's how we can accomplish our single sign on.

isTargetInterceptor is called whenever a user requests access to a protected page. The Application server will run through it's list of installed TAIs (see Part II for installing the TAI) and call isTargetInterceptor on each one. This is the servers way of determining if it should use that TAI for the incoming request. In our simple example, if the request has the parameter customUser, we tell the application server to return true, signaling that we want to use this TAI.

Once the server finds a TAI that returns true for it's isTargetInterceptor method, it will proceed to call that TAI's negotiateValidateandEstablishTrust method. This method is in charge of actually checking whether we want to trust the incoming request, and therefore forward the user to the requested page.

In the example, if the userId is equal to "portalUser", we create a TAIResult with a 200 response, indicating that all is good and the user can continue. Otherwise, the access is forbidden.

Since the url path /myportal is protected once security is configured on WebSphere Portal, all requests sent to /myportal will be challenged against this TAI. If the request included a query parameter such as /myportal?customUser=portalUser, the CustomPortalTAI would be invoked, and the user would be passed to the corresponding page in the portal.

Note that we're not creating any Subject information or anything else to create the user credentials. WebSphere Portal will take care of most of the default creation for you, once the success is found in the TAI.

Monday, April 13, 2009

Funny article on programming progression

Found this online, and although it's a little older, I thought it was pretty funny.

Evolution of a Programmer

Thursday, April 9, 2009

Fixing WPSconfig if cannot find native2ascii

This really only applies to older versions of WebSphere Portal (pre 6.1, as I believe it uses ConfigEngine now), but I've encountered this a couple times for a variety of reasons:

Executing native2ascii with native encoding 'UTF-8': wpconfig.properties -> wpconfig_ascii.properties

java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:85)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:58)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:60)
at java.lang.reflect.Method.invoke(Method.java:391)
at com.ibm.ws.bootstrap.WSLauncher.run(WSLauncher.java:222)
at java.lang.Thread.run(Thread.java:570)
Caused by: java.lang.NoClassDefFoundError: sun/tools/native2ascii/Main
at com.ibm.wps.config.WpsConfigMain.convertToAscii(WpsConfigMain.java:991)
at com.ibm.wps.config.WpsConfigMain.process(WpsConfigMain.java:523)
at com.ibm.wps.config.WpsConfigMain.main(WpsConfigMain.java:204)
... 7 more


Just add the following to the classpath portion of the command at the bottom of the WPSconfig.sh or WPSconfig.bat file:

$JAVA_HOME/lib/tools.jar

The jar tools.jar is missing from the classpath apparently, or buried too far down, and the script is dependent on this to create the OS specific properties file to run the task.

Monday, April 6, 2009

Google Geocoding and Restlet

The other week I needed to code a web page containing interaction between a set of elements and Google Maps, driven off a list of html links in a separate section of the page . The elements had latitude and longitude fields in the database, but this information wasn't completely filled in, especially for internationally based elements.

Given that I needed to be able to click on the links and bring back an individual element (or a subset of elements), I pulled the data out of the database and shoved it in a cache, rather than looking up the data each time (basic stuff).

The problem was, I had about a quarter of the elements lacking the necessary coordinate information to plot on the map.

So the idea was to fill in as much as the missing information at cache load, using Google's Gecoding service. To call the Google Gecoding API, I used the Restlet framework. Code is below, note that not all classes are included (such as model objects and constants files). The cache implementation is Ehcache.

All code stripped down to protect the innocent.

Method to load cache:

public synchronized void load()
{
PropertyDAO propDao = new PropertyDAOImpl(conn);
List<Property> propList = null;

try
{
// get the properties from growth right now
propList = propDao.getPropertiesGeoInfo();
}
catch (SQLException e)
{
e.printStackTrace();
}

if (propList != null)
{
// get the property cache
CacheManager manager = CacheManager.getInstance();
Cache propCache = manager.getCache("myCache");

// for each property, check if it has latitude and longitude, fill in if it doesn't and put in the list
for (Property prop : propList)
{
// check to see if there is a latitude and longitude
if (prop.getLatitudeNbr() == 0 || prop.getLongitudeNbr() == 0)
{
lookupGeoInfo(prop);
}

// add it to the cache
propCache.put(new Element(prop.getFacilityNbr(), prop));
}
}
}
The Property object is the pojo representing the element to be mapped. It comes from the data source with latitude and longitude information, but if it's empty, that's where the Geocoding service comes in:

private void lookupGeoInfo(Property prop)
{
// create a new restlet request
Request request = new Request();
Reference ref = new Reference("http://maps.google.com/maps/geo");

// map the parameters to the geocoding service
mapParams(ref, prop);

request.setResourceRef(ref);
request.setMethod(Method.GET);

Client client = new Client(Protocol.HTTP);

// call Google and get the response back
Response response = client.handle(request);

if (response != null)
{
Representation rep = response.getEntity();

try
{
/***
* split the tokens of the response.
* The format is 4 tokens:
* 200,6,42.7,-73.69
* Status code, accuracy, latitude, longitude
*
**/
String[] tokens = rep.getText().split(",");
// if the response code is success, get the lat and long and set it on the prop object
if ("200".equalsIgnoreCase(tokens[0]))
{
prop.setLatitudeNbr(Double.parseDouble(tokens[2]));
prop.setLongitudeNbr(Double.parseDouble(tokens[3]));
}
}
catch (IOException ie)
{
ie.printStackTrace();
}
}

// explicitly close the call
request.release();
}
Here we create a Restlet Request and set it's endpoint to Google's Geocoding endpoint. Once we've established where the request is going, we set the parameters Google expects (handled by mapParams). Note the format of the response - in this case, I just want the latitude and longitude, so I request the response to come back as csv format and parse it according to it's positions. There are other formats available, check the Geocoding Responses section for more info.

mapParams method:

private void mapParams(Reference ref, Property prop)
{
// flag to know whether to add a comma or not
boolean first = true;

StringBuilder sb = new StringBuilder();
// add address line one if it's there
if (!nullOrEmpty(prop.getAddrLn1Txt()))
first = formatForParam(prop.getAddrLn1Txt(), first, sb);

if (!nullOrEmpty(prop.getAddrLn2Txt()))
first = formatForParam(prop.getAddrLn2Txt(), first, sb);

if (!nullOrEmpty(prop.getCityName()))
formatForParam(prop.getCityName(), first, sb);

if (!nullOrEmpty(prop.getStateCd()))
formatForParam(prop.getStateCd(), first, sb);

if (!nullOrEmpty(prop.getPostalCd()))
formatForParam(prop.getPostalCd(), first, sb);

if (!nullOrEmpty(prop.getCountryName()))
formatForParam(prop.getCountryName(), first, sb);

// now the address parameter built from the prop object
ref.addQueryParameter("q", sb.toString());

// add the sensor param (true for a gps device)
ref.addQueryParameter("sensor", "false");

// the desired output - all we need is the csv, we don't care about other data they send back
ref.addQueryParameter("output", "csv");

// the encoding the response is coming back in
ref.addQueryParameter("oe", "utf8");

// add the key to geo code request
ref.addQueryParameter("key", geoCodeKey);
}
The Reference.addQueryParameter method adds a query string parameter to your outgoing Restlet Request, allowing you to set to set parameters more elegantly than building the whole url in a StringBuilder/StringBuffer. The method attempts to build the address line off of the Property object fields, and then sets all other parameters for the geocoding service appropriately. Note that you'll need a key from Google to perform this call, same as you would with Google Maps.

Putting it all together:

public class PropertyCacheLoad
{
private Connection conn = null;
private String geoCodeKey = null;

public PropertyCacheLoad(Connection conn, String geoCodeKey)
{
this.conn = conn;
this.geoCodeKey = geoCodeKey;
}

/*
*
*/
public void configure(String configFile)
{
CacheManager.create(configFile);
}

/*
*
*/
public synchronized void load()
{
PropertyDAO propDao = new PropertyDAOImpl(conn);

List<property:gt; propList = null;

try
{
// get the properties from growth right now
propList = propDao.getPropertiesGeoInfo();
}
catch (SQLException e)
{
e.printStackTrace();
}

if (propList != null)
{
// get the property cache
CacheManager manager = CacheManager.getInstance();
Cache propCache = manager.getCache("myCache");

// for each property, check if it has latitude and longitude, fill in if it doesn't and put in the list
for (Property prop : propList)
{
// check to see if there is a latitude and longitude
if (prop.getLatitudeNbr() == 0 || prop.getLongitudeNbr() == 0)
{
lookupGeoInfo(prop);
}

// add it to the cache
propCache.put(new Element(prop.getFacilityNbr(), prop));
}
}

}

/**
* @param prop
*/
private void lookupGeoInfo(Property prop)
{
// create a new restlet request
Request request = new Request();
Reference ref = new Reference("http://maps.google.com/maps/geo");

// map the parameters to the geocoding service
mapParams(ref, prop);

request.setResourceRef(ref);
request.setMethod(Method.GET);

Client client = new Client(Protocol.HTTP);

// call Google and get the response back
Response response = client.handle(request);

if (response != null)
{
Representation rep = response.getEntity();

try
{
/***
* split the tokens of the response.
* The format is 4 tokens:
* 200,6,42.7,-73.69
* Status code, accuracy, latitude, longitude
*
**/
String[] tokens = rep.getText().split(",");
// if the response code is success, get the lat and long and set it on the prop object
if ("200".equalsIgnoreCase(tokens[0]))
{
prop.setLatitudeNbr(Double.parseDouble(tokens[2]));
prop.setLongitudeNbr(Double.parseDouble(tokens[3]));
}
}
catch (IOException ie)
{
ie.printStackTrace();
}
}

// explicitly close the call
request.release();
}

/**
* @param ref
* @param prop
*/
private void mapParams(Reference ref, Property prop)
{
// flag to know whether to add a comma or not
boolean first = true;

StringBuilder sb = new StringBuilder();
if (!nullOrEmpty(prop.getAddrLn1Txt()))
first = formatForParam(prop.getAddrLn1Txt(), first, sb);

if (!nullOrEmpty(prop.getAddrLn2Txt()))
first = formatForParam(prop.getAddrLn2Txt(), first, sb);

if (!nullOrEmpty(prop.getCityName()))
formatForParam(prop.getCityName(), first, sb);

if (!nullOrEmpty(prop.getStateCd()))
formatForParam(prop.getStateCd(), first, sb);

if (!nullOrEmpty(prop.getPostalCd()))
formatForParam(prop.getPostalCd(), first, sb);

if (!nullOrEmpty(prop.getCountryName()))
formatForParam(prop.getCountryName(), first, sb);

// now the address parameter built from the prop object
ref.addQueryParameter("q", sb.toString());

// add the sensor param (true for a gps device)
ref.addQueryParameter("sensor", "false");

// the desired output - all we need is the csv, we don't care about other data they send back
ref.addQueryParameter("output", "csv");

// the encoding the response is coming back in
ref.addQueryParameter("oe", "utf8");

// add the key to geo code request
ref.addQueryParameter("key", geoCodeKey);
}

/**
* @param s
* @return
*/
private boolean nullOrEmpty(String s)
{
return (s == null || s.trim().length() == 0);
}

/**
* @param s
* @param first
* @param sb
* @return
*/
private boolean formatForParam(String s, boolean first, StringBuilder sb)
{
if (first)
sb.append(s);
else
sb.append(", " + s);

return false;
}

}