Parallel Programming Issues

• Goal is to reduce execution time
• computation time
• idle time - waiting for data from other processors
• communication time - time the processors take to send and receive messages
  
• Load Balancing
  
• divide the work equally among the available processors
  
• Minimizing Communication
• reduce the number of messages passed
  
• reduce amount of data passed in messages
  
• Where possible - overlap communication and computation
• Many problems scale well to only a limited number of processors