Parallel programming in Java

If you are asking about pure parallel programming i.e. not concurrent programming then you should definitely try MPJExpress http://mpj-express.org/. It is a thread-safe implementation of mpiJava and it supports both distributed and shared memory models. I have tried it and found very reliable.

1 import mpi.*;  
2  
3 
/**  
4  * Compile:impl specific.  
5  * Execute:impl specific.  
6  */  
7  
8 public class Send {  
9 
10     public static void main(String[] args) throws Exception { 
11 
12         MPI.Init(args); 
13 
14         int rank = MPI.COMM_WORLD.Rank() ; //The current process.
15         int size = MPI.COMM_WORLD.Size() ; //Total number of processes
16         int peer ; 
17 
18         int buffer [] = new int[10]; 
19         int len = 1 ;
20         int dataToBeSent = 99 ; 
21         int tag = 100 ; 
22 
23         if(rank == 0) { 
24 
25             buffer[0] = dataToBeSent ; 
26             peer = 1 ; 
27             MPI.COMM_WORLD.Send(buffer, 0, len, MPI.INT, peer, tag) ; 
28             System.out.println("process <"+rank+"> sent a msg to "+ 29                                "process <"+peer+">") ; 
30 
31         } else if(rank == 1) { 
32 
33             peer = 0 ; 
34             Status status = MPI.COMM_WORLD.Recv(buffer, 0, buffer.length, 35                                                 MPI.INT, peer, tag); 
36             System.out.println("process <"+rank+"> recv'ed a msg\n"+ 37                                "\tdata   <"+buffer[0]    +"> \n"+ 38                                "\tsource <"+status.source+"> \n"+ 39                                "\ttag    <"+status.tag   +"> \n"+ 40                                "\tcount  <"+status.count +">") ; 
41 
42         } 
43 
44         MPI.Finalize(); 
45 
46     }  
47 
48 }

One of the most common functionalities provided by messaging libraries like MPJ Express is the support of point-to-point communication between executing processes. In this context, two processes belonging to the same communicator (for instance the MPI.COMM_WORLD communicator) may communicate with each other by sending and receiving messages. A variant of the Send() method is used to send the message from the sender process. On the other hand, the sent message is received by the receiver process by using a variant of the Recv() method. Both sender and receiver specify a tag that is used to find a matching incoming messages at the receiver side.

After initializing the MPJ Express library using the MPI.Init(args) method on line 12, the program obtains its rank and the size of the MPI.COMM_WORLD communicator. Both processes initialize an integer array of length 10 called buffer on line 18. The sender process—rank 0—stores a value of 10 in the first element of the msg array. A variant of the Send() method is used to send an element of the msg array to the receiver process.

The sender process calls the Send() method on line 27. The first three arguments are related to the data being sent. The sending bu!er—the bu!er array—is the first argument followed by 0 (o!set) and 1 (count). The data being sent is of MPI.INT type and the destination is 1 (peer variable); the datatype and destination are specified as fourth and fifth argument to the Send() method. The last and the sixth argument is the tag variable. A tag is used to identify messages at the receiver side. A message tag is typically an identifier of a particular message in a specific communicator. On the other hand the receiver process (rank 1) receives the message using the blocking receive method.

Java supports threads, thus you can have multi threaded Java application. I strongly recommend the Concurrent Programming in Java: Design Principles and Patterns book for that:

http://java.sun.com/docs/books/cp/

You can have a look at Hadoop and Hadoop Wiki.This is an apache framework inspired by google's map-reduce.It enables you to do distributed computing using multiple systems.Many companies like Yahoo,Twitter use it(Sites Powered By Hadoop).Check this book for more information on how to use it Hadoop Book.

You want to look at the Java Parallel Processing Framework (JPPF)