PETSc 2.0 Users Manual

Satish Balay, William Gropp, Lois Curfman McInnes, and Barry Smith
Mathematics and Computer Science Division
Argonne National Laboratory

http://www.mcs.anl.gov/petsc

ANL-95/11 - Revision 2.0.24, April 6, 1999
(This manual is intended for use with PETSc 2.0.24.)

Abstract:

This manual describes the use of PETSc 2.0 for the numerical solution of partial differential equations and related problems on high-performance computers. The Portable, Extensible Toolkit for Scientific Computation (PETSc) is a suite of data structures and routines that provide the building blocks for the implementation of large-scale application codes on parallel (and serial) computers. PETSc 2.0 uses the MPI standard for all message-passing communication.

PETSc includes an expanding suite of parallel linear and nonlinear equation solvers and unconstrained minimization modules that may be used in application codes written in Fortran, C, and C++. PETSc provides many of the mechanisms needed within parallel application codes, such as parallel matrix and vector assembly routines. The library is organized hierarchically, enabling users to employ the level of abstraction that is most appropriate for a particular problem. By using techniques of object-oriented programming, PETSc provides enormous flexibility for users.

PETSc is a sophisticated set of software tools; as such, for some users it initially has a much steeper learning curve than a simple subroutine library. In particular, for individuals without some computer science background or experience programming in C, Pascal, or C++, it may require a significant amount of time to take full advantage of the features that enable efficient software use. However, the power of the PETSc design and the algorithms it incorporates make the efficient implementation of many application codes much simpler than ``rolling them'' yourself. For many simple (or even relatively complicated) tasks a package such as Matlab is often the best tool; PETSc is not intended for the classes of problems for which effective Matlab code can be written.

Since PETSc is still under development, small changes in usage and calling sequences of routines may occur. PETSc is supported; see the readme.html in the PETSc distribution directory or the web site http://www.mcs.anl.gov/petsc for information on contacting support.

Getting Information on PETSc:

On-line:

  • Manual pages on all routines including example usage
  • docs/manualpages/index.html in the distribution or
  • http://www.mcs.anl.gov/petsc/docs/manualpages/index.html

  • Troubleshooting
  • docs/troubleshooting.html in the distribution or
  • http://www.mcs.anl.gov/petsc/docs/troubleshooting.html

    In this manual:
  • Basic introduction: Getting Started
  • Assembling vectors and matrices: Vectors and Matrices
  • Linear solvers: SLES: Linear Equations Solvers
  • Nonlinear solvers: SNES: Nonlinear Solvers and Unconstrained Minimization
  • Timestepping (ODE) solvers: TS: Scalable ODE Solvers
  • Index: Index .

    Acknowledgments:

    We especially thank Victor Eijkhout, David Keyes, and Matthew Knepley for their valuable comments on the source code, functionality, and documentation for PETSc 2.0. In addition, we thank all PETSc users for their many suggestions, bug reports, and encouragement.

    Some of the source code and utilities in PETSc (or software used by PETSc) have been written by


    while visiting Argonne National Laboratory or working with us.

    PETSc uses routines from


    to provide a small subset of its low-level functionality.

    PETSc interfaces to the following external software:


    These are all optional packages and do not need to be installed to use PETSc.

    Contents


    Contents

  • Introduction to PETSc
  • Getting Started
  • Suggested Reading
  • Running PETSc Programs
  • Writing PETSc Programs
  • Simple PETSc Examples
  • Include Files
  • The Options Database
  • Vectors
  • Matrices
  • Linear Solvers
  • Error Checking
  • Parallel Programming
  • Compiling and Running Programs
  • Writing Application Codes with PETSc
  • Referencing PETSc
  • Directory Structure
  • Programming with PETSc
  • Vectors and Distributing Parallel Data
  • Creating and Assembling Vectors
  • Basic Vector Operations
  • Indexing and Ordering
  • Application Orderings
  • Local to Global Mappings
  • Structured Grids Using Distributed Arrays
  • Creating Distributed Arrays
  • Local/Global Vectors and Scatters
  • Grid Information
  • Software for Managing Unstructured Grids
  • Index Sets
  • Scatters and Gathers
  • Scattering Ghost Values
  • Vectors with Locations for Ghost Values
  • Matrices
  • Creating and Assembling Matrices
  • Sparse Matrices
  • Sequential AIJ Sparse Matrices
  • Preallocation of Memory for Sequential AIJ Sparse Matrices
  • Parallel AIJ Sparse Matrices
  • Preallocation of Memory for Parallel AIJ Sparse Matrices
  • Dense Matrices
  • Basic Matrix Operations
  • Matrix-Free Matrices
  • Other Matrix Operations
  • Partitioning
  • SLES: Linear Equations Solvers
  • Using SLES
  • Solving Successive Linear Systems
  • Krylov Methods
  • Preconditioning within KSP
  • Convergence Tests
  • Convergence Monitoring
  • Understanding the Operator's Spectrum
  • Other KSP Options
  • Preconditioners
  • ILU and ICC Preconditioners
  • SOR and SSOR Preconditioners
  • LU Factorization
  • Block Jacobi and Overlapping Additive Schwarz Preconditioners
  • Shell Preconditioners
  • Combining Preconditioners
  • Multigrid Preconditioners
  • SNES: Nonlinear Solvers and Unconstrained Minimization
  • Basic Usage
  • Solving Systems of Nonlinear Equations
  • Solving Unconstrained Minimization Problems
  • The Nonlinear Solvers
  • Line Search Techniques
  • Trust Region Methods
  • General Options
  • Convergence Tests
  • Convergence Monitoring
  • Checking Accuracy of Derivatives
  • Inexact Newton-like Methods
  • Matrix-Free Methods
  • Finite Difference Jacobian Approximations
  • TS: Scalable ODE Solvers
  • Basic Usage
  • Solving Time-dependent Problems
  • Using PVODE from PETSc
  • Solving Steady-State Problems with Pseudo-Timestepping
  • Graphics
  • Windows as Viewers
  • Simple Drawing
  • Line Graphs
  • Graphical Convergence Monitor
  • Disabling Graphics at Compile Time
  • PETSc Fortran Users
  • Differences between PETSc Interfaces for C and Fortran
  • Include Files
  • Error Checking
  • Array Arguments
  • Calling Fortran Routines from C (and C Routines from Fortran)
  • Passing Null Pointers
  • Duplicating Multiple Vectors
  • Matrix and Vector Indices
  • Setting Routines
  • Compiling and Linking Fortran Programs
  • Routines with Different Fortran Interfaces
  • Fortran90
  • Sample Fortran77 Programs
  • Additional Information
  • Profiling
  • Basic Profiling Information
  • Interpreting -log_summary Output: The Basics
  • Interpreting -log_summary Output: Parallel Performance
  • Using -log and -log_all with PETScView
  • Using -log_mpe with Upshot/Nupshot
  • Profiling Application Codes
  • Profiling Multiple Sections of Code
  • Restricting Event Logging
  • Interpreting -log_info Output: Informative Messages
  • Time
  • Saving Output to a File
  • Accurate Profiling: Overcoming the Overhead of Paging
  • Hints for Performance Tuning
  • Compiler Options
  • Profiling
  • Aggregation
  • Efficient Memory Allocation
  • Sparse Matrix Assembly
  • Sparse Matrix Factorization
  • PetscMalloc() Calls
  • Data Structure Reuse
  • Numerical Experiments
  • Tips for Efficient Use of Linear Solvers
  • Detecting Memory Allocation Problems
  • Machine-Specific Optimizations
  • System-Related Problems
  • Other PETSc Features
  • Runtime Options
  • The Options Database
  • User-Defined Options
  • Keeping Track of Options
  • Viewers: Looking at PETSc Objects
  • Debugging
  • Error Handling
  • Incremental Debugging
  • Complex Numbers
  • Emacs Users
  • Parallel Communication
  • Makefiles
  • Our Makefile System
  • Makefile Commands
  • Customized Makefiles
  • PETSc Flags
  • Sample Makefiles
  • Limitations
  • Advanced Features of Matrices and Solvers
  • Extracting Submatrices
  • Matrix Factorization
  • Unimportant Details of KSP
  • Unimportant Details of PC
  • Index
  • Bibliography