How to perform distributed computations with F# (.Net)
There is a project called MBrace which does pretty much exactly what you describe :-).
It lets you write cloud computations using the cloud
block:
let first = cloud { return 15 }
let second = cloud { return 27 }
You can compose them using let!
as with asynchronous workflows and you can also create them from asynchronous workflows using Cloud.ofAsync
. Cloud computations can be distributed over network using Cloud.Parallel
:
cloud {
let! results = [ first; second ] |> Cloud.Parallel
return List.sum results }
Currently, there are MBrace bindings for running the computation locally (for testing) and inside Azure cluster, but there is some work-in-progress on supporting Amazon too.
For more information see mbrace.io and there is also a nice talk from Mathias Brandewinder on crunching through big data with MBrace
fsharp.org has a page dealing with Cloud Data, Compute and Messaging with F#, it provides up-to-date ressources on the matter.
As Tomas Petricek said, MBrace seems to be the idiomatic way to do distributed computation in F#. Sadly it focuses on cloud computing (Azure and Amazon) and provides very little information on local multi-machine clusters. I found a thread that deals with the subject and appears to provide a solution but an official tutorial (and maybe some build-in functions) would be nice.
Microsoft's Prajna was developed in F# and offers an alternative to Sparks. The documentation explains how to build local multi-machine clusters (windows machine only it seems). It might be the easiest solution but it seems to be dead.
Another option might be akka.net which has an F# API.
I definitely recommend you akka.net. I'm currently implementing distributed integration solutions using it and can tell you this is awesome. Project Orleans by Microsoft Research is also pretty good, although it's not idiomatic f# approach