guycarre is right in that this is a gnome/unity problem most likely. I can confirm Ubuntu server 14.04.1 does not have this problem.
-- You received this bug notification because you are a member of Ubuntu Touch seeded packages, which is subscribed to openssh in Ubuntu. https://bugs.launchpad.net/bugs/1380084 Title: SSH client requires SSH_AUTH_SOCK=0 will not work without it, can't use key auth without SSH_AUTH_SOCK=0 Status in “openssh” package in Ubuntu: Confirmed Bug description: I expect that when using key based authentication I do $ssh user@0.0.0.0 I expect to be asked for the key password. Instead I get Agent admitted failure to sign using the key. Permission denied (publickey). Using SSH_AUTH_SOCK=0 solves the problem allowing me to be asked for the key password. What should happen is that I should not have to specify SSH_AUTH_SOCK=0 on the CLI, I should only have to type $ user@0.0.0.0 ProblemType: Bug DistroRelease: Ubuntu 14.04 Package: openssh-client 1:6.6p1-2ubuntu2 ProcVersionSignature: Ubuntu 3.13.0-37.64-generic 3.13.11.7 Uname: Linux 3.13.0-37-generic i686 NonfreeKernelModules: wl ApportVersion: 2.14.1-0ubuntu3.5 Architecture: i386 CurrentDesktop: Unity Date: Sat Oct 11 07:19:08 2014 EcryptfsInUse: Yes InstallationDate: Installed on 2014-09-08 (33 days ago) InstallationMedia: Ubuntu 14.04.1 LTS "Trusty Tahr" - Release i386 (20140722.2) ProcEnviron: LANGUAGE=en_US PATH=(custom, no user) XDG_RUNTIME_DIR=<set> LANG=en_US.UTF-8 SHELL=/bin/bash RelatedPackageVersions: ssh-askpass N/A libpam-ssh N/A keychain N/A ssh-askpass-gnome 1:6.6p1-2ubuntu2 SSHClientVersion: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2, OpenSSL 1.0.1f 6 Jan 2014 SourcePackage: openssh UpgradeStatus: No upgrade log present (probably fresh install) To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1380084/+subscriptions -- Mailing list: https://launchpad.net/~touch-packages Post to : touch-packages@lists.launchpad.net Unsubscribe : https://launchpad.net/~touch-packages More help : https://help.launchpad.net/ListHelp