Distributed arrays ( DAs), which are used in conjunction with PETSc vectors, are intended for use with logically regular rectangular grids when communication of nonlocal data is needed before certain local computations can occur. PETSc distributed arrays are designed only for the case in which data can be thought of as being stored in a standard multidimensional array; thus, DAs are not intended for parallelizing unstructured grid problems, etc. DAs are intended for communicating vector (field) information; they are not intended for storing matrices.
For example, a typical situation one encounters in solving PDEs in parallel is that, to evaluate a local function, f(x), each processor requires its local portion of the vector x as well as its ghost points (the bordering portions of the vector that are owned by neighboring processors). Figure 8 illustrates the ghost points for the seventh processor of a two-dimensional, regular parallel grid. Each box represents a processor; the ghost points for the seventh processor's local part of a parallel array are shown in gray.