Clarification of the leading dimension in CUBLAS when transposing

If you are using row-major representation then the number of "columns" will be leading dimension and vice versa in column-major representation number of "rows".

Nothing changes. The leading dimension always refers to the length of the first dimension of the array. The data order flags (normal, transpose, conjugate) only indicate to BLAS how the data within the array is stored. They have no effect on the array itself, which is always column major ordered and requires an LDA value for indexing in 2D.

So whether the matrix data is stored in transposed form or not, an m x n array always has LDA>=m.