What does it mean for Redis to be an "in memory" database?

Redis is an in-memory but persistent on disk database, so it represents a different trade off where very high write and read speed is achieved with the limitation of data sets that can't be larger than memory. Another advantage of in memory databases is that the memory representation of complex data structures is much simpler to manipulate compared to the same data structure on disk, so Redis can do a lot, with little internal complexity. At the same time the two on-disk storage formats (RDB and AOF) don't need to be suitable for random access, so they are compact and always generated in an append-only fashion (Even the AOF log rotation is an append-only operation, since the new version is generated from the copy of data in memory).

http://redis.io/topics/faq

In redis, all data has to be in memory. This is the point which is totally different from other NoSQL. Usually when you access and read some data in a database, you don't know if the data is in memory (cache) or not, but in the case of Redis, it's guaranteed that all data is in memory. Writing to disk is optional, which you can think of as having the trunk on memory and a kind of backup on the disk. You may lose data which is saved after last saving to a disk if you suddenly turn off the server.

And of course the advantage of it is a performance. Since all data is in RAM, it's incredibly fast.

They are not mutually exclusive. In-memory means all the data is stored in memory for it to be accessed. It doesn't mean that it should be impossible to also store it on to the disk from time to time, but it definitely shouldn't be accessed from the disk unless some exceptional events occur. When data is read, it can either be read from the disk or from memory. In Redis's case, it's always retrieved from memory (hence - in-memory database). Writing the data onto the disk every two seconds is useful for having a backup in case of outage. On one end, the users accessing the database are accessing the data stored in memory, also, the backup mechanism accesses the data from the memory and writes it to the disk. In case of a system failure, the data stored in memory is lost. But when booting up, the data (up to the last 2 seconds) is retrieved from the disk and stored into memory again for the applications to use it.