Today I learned that ssh will not work with public key access if the .ssh/authorized_keys file on the server is set to read by group/other. I chmod'd it 700 and it worked.