How to Run Multiple Test Files with Haskell Stack Project
Here is a solution with just stack and HUnit. Nothing against hspec, htf, tasty, etc etc, but likewise there is not much glue needed even without those, if you're already using HUnit. It doesn't require editing the cabal file. The original question implies use of hspec, so the @epsilonhalbe is still closer on that criteria.
.
├─stack.yaml
├─package.yaml
├─src/
| ├─A.hs
| ├─B.hs
| ├─Main.hs
├─tst/
| ├─ATest.hs
| ├─BTest.hs
| ├─Main.hs
Example package.yaml
file:
name: example
version: 0.1.0.0
dependencies:
- HUnit >= 1.6.1.0 && < 2
library:
source-dirs: src
executables:
example-exe:
main: Main.hs
source-dirs: src
dependencies:
- example
tests:
example-test:
main: Main.hs
source-dirs: tst
dependencies:
- example
- HUnit
In ATest.hs and BTest.hs declare list of tests called huTests
in the usual HUnit way, eg,
huTests = ["egTest" ~: "a" ~=? "a"]
.
Then tst/Main.hs
has the glue in a common HUnit idiom (see eg this answer):
import ATest (huTests)
import BTest (huTests)
import System.Exit
import Test.HUnit
main :: IO ()
main = do
results <- runTestTT $
test (ATest.huTests ++ BTest.huTests)
if errors results + failures results == 0 then
putStrLn "Tests passed."
else
die "Tests failed."
Here is a setup for a directory structure like this
> tree
.
├── example.cabal
├── app
│ └── Main.hs
├── ChangeLog.md
├── LICENSE
├── Setup.hs
├── src
│ ├── A
│ │ └── C.hs
│ ├── A.hs
│ └── B.hs
├── stack.yaml
└── tst
├── integration
│ └── Spec.hs
└── unit
├── A
│ └── CSpec.hs
├── ASpec.hs
├── BSpec.hs
└── Spec.hs
you want to have integration tests that are separate from the usual unit tests and several sub-modules that correspond to each module in your src
-folder
first of all you need to add the test suites to your
example.cabal
file
name: example
...
-- copyright:
-- category:
build-type: Simple
extra-source-files: ChangeLog.md
cabal-version: >=1.10
executable testmain
main-is: Main.hs
hs-source-dirs: app
build-depends: base
, example
library
exposed-modules: A.C,A,B
-- other-modules:
-- other-extensions:
build-depends: base >=4.9 && <4.10
hs-source-dirs: src
default-language: Haskell2010
test-suite unit-tests
type: exitcode-stdio-1.0
main-is: Spec.hs
hs-source-dirs: tst/unit
build-depends: base
, example
, hspec
, hspec-discover
, ...
test-suite integration-tests
type: exitcode-stdio-1.0
main-is: Spec.hs
hs-source-dirs: tst/integration
build-depends: base
, example
, hspec
, ...
put the following in your tst/unit/Spec.hs
it is from hspec-discover
and it discovers (hence the name) all modules of the form ...Spec.hs
and executes the spec
function from each of those modules.
tst/unit/Spec.hs
{-# OPTIONS_GHC -F -pgmF hspec-discover #-}
just this single line
Other test files
then add your unit tests in your ASpec.hs
, and others in BSpec.hs
,CSpec.hs
and your Spec.hs
in the tst/integration
folder
module ASpec where
import Test.Hspec
import A
spec :: Spec
spec = do
describe "Prelude.head" $ do
it "returns the first element of a list" $ do
head [23 ..] `shouldBe` (23 :: Int)
it "returns the first element of an *arbitrary* list" $
property $ \x xs -> head (x:xs) == (x :: Int)
it "throws an exception if used with an empty list" $ do
evaluate (head []) `shouldThrow` anyException
you can then compile and run your tests with
$> stack test
# now all your tests are executed
$> stack test :unit-tests
# now only the unit tests run
$> stack test :integration-tests
# now only the integration tests run
Sources
You can find all the examples at https://hspec.github.io, if you want to know more about hspec-style testing I guess it would be best to start there. For the stack - go to https://haskellstack.org - there is some information about testing/benchmarking there - I mean about running tests and benchmarks.
For different testing style in haskell see HUnit, QuickCheck, Smallcheck, doctests (If I forgot one, my dearest apologies - those are the ones that I use regularly as well).