Parallel Computing

Parallel Computing

What is Parallel Computing?

• Traditionally, software has been written for serial computation:
• To be run on a single computer having a single Central Processing Unit (CPU);
• A problem is broken into a discrete series of instructions.
• Instructions are executed one after another.
• Only one instruction may execute at any moment in time.

In the simplest sense, parallel computing is the simultaneous use of multiple compute resources to solve a computational problem:

• To be run using multiple CPUs
• A problem is broken into discrete parts that can be solved concurrently
• Each part is further broken down to a series of instructions
• Instructions from each part execute simultaneously on different CPUs

• The compute resources might be:
• A single computer with multiple processors;
• An arbitrary number of computers connected by a network;
• A combination of both.
• The computational problem should be able to:
• Be broken apart into discrete pieces of work that can be solved simultaneously;
• Execute multiple program instructions at any moment in time;
• Be solved in less time with multiple compute resources than with a single compute resource.

The Universe is Parallel:

• Parallel computing is an evolution of serial computing that attempts to emulate what has always been the state of affairs in the natural world: many complex, interrelated events happening at the same time, yet within a temporal sequence.

 Uses for Parallel Computing:

• Science and Engineering: Historically, parallel computing has been considered to be "the high end of computing", and has been used to model difficult problems in many areas of science and engineering:

o    Atmosphere, Earth, Environment o    Physics - applied, nuclear, particle, condensed matter, high pressure, fusion, photonics o    Bioscience, Biotechnology, Genetics o    Chemistry, Molecular Sciences

o    Geology, Seismology o    Mechanical Engineering - from prosthetics to spacecraft o    Electrical Engineering, Circuit Design, Microelectronics o    Computer Science, Mathematics