4.4.3. Grid Information

Up: Contents Next: Software for Managing Unstructured Grids Previous: Local/Global Vectors and Scatters

The global indices of the lower left corner of the local portion of the array as well as the local array size can be obtained with the commands

   ierr = DAGetCorners(DA da,int *x,int *y,int *z,int *m,int *n,int *p); 
   ierr = DAGetGhostCorners(DA da,int *x,int *y,int *z,int *m,int *n,int *p); 
The first version excludes any ghost points, while the second version includes them. The routine DAGetGhostCorners() deals with the fact that subarrays along boundaries of the problem domain have ghost points only on their interior edges, but not on their boundary edges.

When either type of stencil is used, DA_STENCIL_STAR or DA_STENCIL_BOX, the local vectors (with the ghost points) represent rectangular arrays, including the extra corner elements in the DA_STENCIL_STAR case. This configuration provides simple access to the elements by employing two- (or three-) dimensional indexing. The only difference between the two cases is that when DA_STENCIL_STAR is used, the extra corner components are not scattered between the processors and thus contain undefined values that should not be used.

To assemble global stiffness matrices, one needs either

1) to be able to determine the global node number of each local node including the ghost nodes. The number may be determined by using the command

   ierr = DAGetGlobalIndices(DA da,int *n,int **idx); 
The output argument n contains the number of local nodes, including ghost nodes, while idx contains a list of the global indices that correspond to the local nodes. Note that the Fortran interface differs slightly; see Section Array Arguments for details.

2) or to set up the vectors and matrices so that their entries may be added using the local numbering. This is done by first calling

   ierr = DAGetISLocalToGlobalMapping(DA da,ISLocalToGlobalMapping *map); 
followed by
   ierr = VecSetLocalToGlobalMapping(Vec x,ISLocalToGlobalMapping map); 
   ierr = MatSetLocalToGlobalMapping(Vec x,ISLocalToGlobalMapping map); 
Now entries may be added to the vector and matrix using the local numbering and VecSetValuesLocal() and MatSetValuesLocal().

Since the global ordering that PETSc uses to manage its parallel vectors (and matrices) does not usually correspond to the ``natural'' ordering of a two- or three-dimensional array, the DA structure provides an application ordering AO (see Section Application Orderings ) that maps between the natural ordering on a rectangular grid and the ordering PETSc uses to parallize. This ordering context can be obtained with the command

   ierr = DAGetAO(DA da,AO *ao); 
In Figure 9 we indicate the orderings for a two-dimensional distributed array, divided among four processors.


Figure 9: Natural Ordering and PETSc Ordering for a 2D Distributed Array (Four Processors)

The example ${}PETSC_DIR/src/snes/examples/tutorials/ex5.c, illustrates the use of a distributed array in the solution of a nonlinear problem. The analogous Fortran program is ${}PETSC_DIR/src/snes/examples/tutorials/ex5f.F; See Chapter SNES: Nonlinear Solvers and Unconstrained Minimization for a discussion of the nonlinear solvers.


Up: Contents Next: Software for Managing Unstructured Grids Previous: Local/Global Vectors and Scatters