How to check whether the folder is created, and if not, create it?

With[{dirname = FileNameJoin[{NotebookDirectory[], "mydir"}]},
 Switch[FileType[dirname],
  None, CreateDirectory[dirname] (* create dir *),
  Directory, Null (* do nothing *),
  File, Print["File with same name already exists!!"] (* error! *)
 ]
]

Ref: FileType, CreateDirectory

Ideally, also check if CreateDirectory succeeded.

As noted by jkuczm in a comment, we can attempt to create the directory unconditionally but tolerate the case that the directory already exists:

Quiet[
  CreateDirectory[{FileNameJoin[NotebookDirectory[], "mydir"}]]
, CreateDirectory::filex
]

This approach differs from the test-then-create approach in that it precludes the possibility of another process sneaking a file operation between the testing and creation. Of course, this possibility is not relevant for typical single-user operation.

If rather than leaving the directory empty we wish to immediately create a contained file, we can create both the directory (as needed) and the file in a single step:

CreateFile[
  FileNameJoin[{NotebookDirectory[], "mydir", "myfile"}]
, CreateIntermediateDirectories -> True
]