Does this implementation of mutex locks result in undefined behavior?

Answering your questions,

  1. If main tries to lock lock[0] twice it should deadlock.

Yes, it would. Unless you use recursive mutexes, but then your child thread would never be able to lock the mutex as main would always have it locked.

  1. extra unlocking lock[0], which was locked by main, should be undefined behavior.

Per the POSIX documentation for pthread_mutex_unlock(), this is undefined behavior for a NORMAL and non-robust mutex. However, the DEFAULT mutex does not have to be NORMAL and non-robust so there is this caveat:

If the mutex type is PTHREAD_MUTEX_DEFAULT, the behavior of pthread_mutex_lock() [and pthread_mutex_unlock()] may correspond to one of the three other standard mutex types as described in the table above. If it does not correspond to one of those three, the behavior is undefined for the cases marked.

(Note my addition of pthread_mutex_unlock(). The table of mutex behavior clearly shows that unlock behavior for a non-owner varies between different types of mutexes and even uses the same "dagger" mark in the "Unlock When Not Owner" column as used in the "Relock" column, and the "dagger" mark refers to the footnote I quoted.)

A robust NORMAL, ERRORCHECK, or RECURSIVE mutex will return an error if a non-owning thread attempts to unlock it, and the mutex remains locked.

A simpler solution is to use a pair of semaphores (the following code is deliberately missing error checking along with empty lines that would otherwise increase readability in order to eliminate/reduce any vertical scroll bar):

#include <semaphore.h>
#include <pthread.h>
#include <stdio.h>
sem_t main_sem;
sem_t child_sem;
void *child( void *arg )
{
    for ( ;; )
    {
        sem_wait( &child_sem );
        sleep( 2 );
        sem_post( &main_sem );
    }
    return( NULL );
}
int main( int argc, char **argv )
{
    pthread_t child_tid;
    sem_init( &main_sem, 0, 0 );
    sem_init( &child_sem, 0, 0 );
    pthread_create( &child_tid, NULL, child, NULL );
    int x = 0;
    for ( ;; )
    {
        // tell the child thread to go
        sem_post( &child_sem );
        // wait for the child thread to finish one iteration
        sem_wait( &main_sem );
        x++;
        printf("%d\n", x);
    }
    pthread_join( child_tid, NULL );
}

The sane thread-safe solution is a condition variable:

//main thread
while(1) {
    x += 1;
    printf("%d\n", x);

    pthread_mutex_lock(&lock);
    pthread_cond_wait(&cond, &lock);
    pthread_mutex_unlock(&lock);
}

then in the sleeper thread you do:

//sleeper thread
while(1) {
    pthread_cond_signal(&cond);
    sleep(2);
}

However you can also read the current time from the operating system and the sleep for the remaining time until next epoch using the high resolution sleep and time.

The next option is using a timerfd to wake you up in a fixed interval. And it can let you know if you missed a wake up.