RxJS 6 - Do I need to unsubscribe when using first in a pipe?

Both RxJS 5 and RxJS 6 versions of first work the same so you don't need to unsubscribe because it completes the chain and thus triggers dispose handlers.

If you want to be sure you can add complete callback to your tap and see if it gets called (you could add it to subscribe as well):

asyncAction$
    .clientLogin()
    .pipe(
        first(),
        tap({
            next: val => console.log(`Test: ${val}`),
            complete: () => console.log(`Complete`),
        }),
    )
    .subscribe();