Unable to resolve symbol: thrown?

thrown? is a special assertion that must show up after is, so you can't nest it in other expressions, so in the context of the is macro, the second assertion will not understand the symbol thrown?.

You could just say:

(deftest main-test
  (is (thrown? Exception (throw (Exception. "stuff"))))
  (is (= 5 (+ 2 3))))

If an exception is thrown in (+ 2 3), clojure.test will report 1 :error and 0 :fail and dump the stack trace.

Also note that your (throw Exception "stuff") is incorrect - you need to construct the Exception correctly inside the throw.

is is a macro that looks for the symbol thrown? in its body and build tests. thrown? is not actually a function you can call. The default behaviour of is fails the test if an exception is thrown that was not beeing looked for, so you can just remove the (not (thrown? from the above example and get the result you are looking for.