I recently switched my Java versions from 1.5 to 1.6 and I’ve been hitting some issues with the java path.
I tried setting the environment variable JAVA_HOME=/usr and that worked for some stuff but I still saw this error when running java.
eg:
>java -version
Unable to find a $JAVA_HOME at "/usr", continuing with system-provided Java

I solved the problem by doing the following.
First I found out where java actually was with:
>ls -l' `which java`
/usr/bin/java -> /System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java

I noticed a command called java_home in that same directory. Running that command yields the correct java home directory.
>/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java_home
/System/Library/Frameworks/JavaVM.framework/Versions/1.6.0/Home

To correctly set the my JAVA_HOME environment variable in future, I added the following to my ~/.profile

export JAVA_HOME=`/System/Library/Frameworks/JavaVM.framework/Versions/Current/Commands/java_home`

Once you have done this you can restart your terminal window or run >. ~/.profile to reload you profile into the current terminal session.