Not able to ssh to another computer, but can ping it?

The server is either not running sshd (and hence not listening on port 22) or has a firewall blocking port 22 (the default ssh port), or in incredibly rare cases running ssh on some other port (which is almost certainly not the case).

First check to make sure sshd is installed (using debian examples)

sudo apt-get install openssh-server

And if so, is it running:

ps -ef | grep sshd

then check to see if it is listening to port 22

sudo netstat -nlp | grep :22
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      946/sshd
tcp6       0      0 :::22                   :::*                    LISTEN      946/sshd

then check your firewall rules (this varies significantly, so I'll show a debian/ubuntu/etc example):

sudo ufw status

sudo ufw show listening
tcp:
  22 * (sshd)
  24224 * (ruby)
tcp6:
  22 * (sshd)
  8080 * (java)
udp:
  123 10.X.Y.Z (ntpd)
  123 * (ntpd)
  18649 * (dhclient)
  24224 * (ruby)
  34131 * (ruby)
  60001 10.87.43.24 (mosh-server)
  68 * (dhclient)
udp6:
  123 fe80::1031:AAAA:BBBB:CCCC (ntpd)
  123 * (ntpd)
  48573 * (dhclient)

If ufw shows it as closed then run (again a debian/ubuntu example)

sudo ufw allow 22

Kind of a weird shot-in-the-dark, but make sure your IP didn't change. I had this issue once - I set a .bashrc alias alias sshdev='ssh [email protected]' as my typical way of logging in, and one day I started getting the following error:

ME-M-216C:~ me$ sshdev 
ssh: connect to host 123.2.3.4 port 22: Connection refused

We just had a power outage at work which reset the IP's, so I was successfully pinging an IP but it wasn't the correct machine. You can use nslookup <IP> to make sure it's the correct machine name that you're trying to ssh into.