OpenSSL CA and non CA certificate

Did you try with basicConstraints=critical,CA:false?

PS: The 'ca' utility doesn't currently check the validity of the CA certificate it is signing with. So it will happily sign with an invalid CA but the verification routines will reject it.

The following, while a bit verbose, should work; can you try it on your setup and report any issues?

#!/bin/sh
#SSLEAY_CONFIG="-config yourfile.cnf"
ROOTCA_SUBJ="-subj /C=US/ST=abc/L=abc/O=test/OU=mine/CN=RootCA/[email protected]"
CA_SUBJ="-subj /C=US/ST=abc/L=abc/O=test/OU=mine/CN=CA/[email protected]"
CERT_SUBJ="-subj /C=US/ST=abc/L=abc/O=test/OU=mine/CN=cert/[email protected]"
ROOTCA_PASS="pass:test"
CA_PASS="pass:test"
CERT_PASS="pass:test"
DIR="demoCA"
mkdir "$DIR" "$DIR"/certs "$DIR"/crl "$DIR"/newcerts "$DIR"/private
touch "$DIR"/index.txt
echo 01 > "$DIR"/crlnumber

# create Root CA
mkdir rootCA rootCA/certs rootCA/crl rootCA/newcerts rootCA/private
openssl req $SSLEAY_CONFIG -new -keyout rootCA/private/rootCAkey.pem -out rootCA/rootCAreq.pem $ROOTCA_SUBJ -passout "$ROOTCA_PASS"
openssl ca $SSLEAY_CONFIG -create_serial -out rootCA/rootCAcert.pem -days 1095 -batch -keyfile rootCA/private/rootCAkey.pem -passin "$ROOTCA_PASS" -selfsign -extensions v3_ca -infiles rootCA/rootCAreq.pem

# create Intermediate CA
mkdir CA CA/certs CA/crl CA/newcerts CA/private
openssl req $SSLEAY_CONFIG -new -keyout CA/private/CAkey.pem -out CA/CAreq.pem -days 365 $CA_SUBJ -passout "$CA_PASS"
openssl ca $SSLEAY_CONFIG -cert rootCA/rootCAcert.pem -keyfile rootCA/private/rootCAkey.pem -passin "$ROOTCA_PASS" -policy policy_anything -out CA/CAcert.pem -extensions v3_ca -infiles CA/CAreq.pem

# create Final Cert
mkdir cert cert/private
openssl req $SSLEAY_CONFIG -new -keyout cert/private/certkey.pem -out cert/certreq.pem -days 365 $CERT_SUBJ -passout "$CERT_PASS"
openssl ca $SSLEAY_CONFIG -cert CA/CAcert.pem -keyfile CA/private/CAkey.pem -passin "$CA_PASS" -policy policy_anything -out cert/cert.pem -infiles cert/certreq.pem
cat rootCA/rootCAcert.pem CA/CAcert.pem > myCA.pem
openssl verify -CAfile myCA.pem cert/cert.pem

When you run openssl verify, -CAfile implies you trust this certificate. So -CAfile should only contain the root-CA, and optionally the intermediate-CA; else it will always return OK. Other certificates should be added through the -untrusted option.

Therefore, if you give it a cert2.pem certificate, signed by the final cert.pem certificate, you'll get:

$ cat CA/CAcert.pem cert/cert.pem > notrust.pem
$ openssl verify -CAfile myCA.pem -untrusted notrust.pem cert2/cert2.pem
    cert2/cert2.pem: C = US, ST = abc, L = abc, O = test, OU = mine, CN = cert, emailAddress = [email protected]
    error 24 at 1 depth lookup:invalid CA certificate

As stated before, the CA utility will happily sign a request with any certificate, even if it is not a CA certificate. Apparently, some older browsers (at a minimum, Firefox versions prior to 33) would even accept such chains.

To find out if your certificate has the isCA bit set, run:

openssl x509 -text -noout -in your_cert_file.crt

In the output, look for the following:

X509v3 Basic Constraints: 
    CA:TRUE

This is a CA certificate. A non-CA cert would have CA:FALSE (or not have the extension at all).

Caveat: you need to include these extensions in your request AND make sure the CA does not override them when signing the request.