Is HMAC-SHA256 and HMAC-SHA512 able to act as drop in replacement for HMAC-SHA1 in HOTP/TOTP?

It's not only the code sample and the test values, it's even there explicitly in the RFC 6238:

TOTP implementations MAY use HMAC-SHA-256 or HMAC-SHA-512 functions, based on SHA-256 or SHA-512 [SHA2] hash functions, instead of the HMAC-SHA-1 function that has been specified for the HOTP computation in [RFC4226].

So, yes.

My inexperienced but slightly educated guess would be that there's nothing wrong with using them even with HOTP (even though the RFC doesn't explicitly say it). After all, HOTP and TOPT are virtually the same.

Obviously, these two hash algorithm aren't supported by the standard. In fact, SHA-1 is used pretty much everywhere in RFC 4226 so, if you replace it with another hash, you'll be implementing a different system.

Furthermore, I wonder what would be the idea behind this: the implementation calls for the result of the HMAC to be truncated and the initial HMAC uses a very short secret: using a different hash algorithm here wouldn't improve security but it would make it incompatible with existing applications and tokens.

edit: RFC 4226 is pretty clear about what algorithm you can use: HMAC-SHA-1 (section 5.2) so, if you must support RFC 4226, you can't use anything else.

RFC 6238, on the other hand, opens the door to the use of SHA-256 and SHA-512 (in section 1.2).

Therefore, it looks like it all depends on your requirements: if you're implementing RFC4226 (HOTP) or must be compatible with it, you cannot use anything but HMAC-SHA-1. If, however, you're implementing RFC 6238 (TOTP), using HMAC-SHA-256 or HMAC-SHA-512 is within the scope of the standard.