Portable, Extensible Toolkit for Scientific Computation

Installation Instructions for PETSc 2.0

http://www.mcs.anl.gov/petsc/

Welcome to PETSc 2.0. Please let us know of any difficulties in installing and using the software. See the file docs/bugreporting.html   for information on reporting bugs and installation problems. Installing PETSc can be the hardest part of using the software, since each machine's configuration is slightly different. If problems crop up, first, consult

the troubleshooting.html guide and
http://www.mcs.anl.gov/petsc/petsc-patches.html

through which most users can solve their problems. We are always willing to assist with an installation if a problem remains unresolved. If installing PETSc has taken more than a few hours and you've gone through the troubleshooting.html guide, please refer to bugreporting.html. We will help everyone with installation problems on any of the machines listed below.

This release has been tested on

• Sun Sparcstations (sun4)
• IBM RS6000 (including SP) (rs6000) (rs6000_sp for SP) (rs6000_gnu for Gnu compilers)(rs6000_64 for Power3 processors in 64 bit mode)
• SGI Workstations (IRIX )
• SGI Power Challenge (IRIX64) (IRIX for 32 bit mode)
• SGI Origin (IRIX64)(IRIX for 32 bit mode)
• Dec Alpha running OSF/ (alpha)
• HP (including Convex Exemplar) (hpux)
• Sun Sparcstations running Solaris (solaris) (solaris_gnu for Gnu compilers)
• Solaris on Intel (solaris_x86)
• Cray T3E (t3e)
• Linux on Intel (linux)
• Windows NT or Windows 95; (win32 for Microsoft developers studio compilers) (win32_gnu for Gnu)
• FreeBSD on Intel (freebsd)

On variants of the OS on the above machines, a little tweaking may be necessary. This information is usually present in the files petsc/bmake/${PETSC_ARCH}/base_variables (e.g., changes in compiler options, libraries supplied by the OS), or in comments within any source files that may fail to compile, or in the PETSc bug fixes file, http://www.mcs.anl.gov/petsc/petsc-patches.html

Separate installation instructions are below for UNIX and Window plaftorms.

Installation under UNIX

PETSc 2.0 requires a MINIMUM of 25 megabytes of disk space to install for a single operating system. The UNIX command "df" can be used to determine the amount of available disk space. See the PETSc FAQ for tips on reducing required space.

PETSc MUST be compiled with an ANSI C compiler (or C++ compiler). Many older Sun workstations have the Sun-bundled C compiler, which is NOT ANSI C and cannot be used. The Gnu compiler gcc can often be used as a replacement for systems that do not have a native ANSI C compiler.

Required Software:

Prior to installing PETSc 2.0, the machine must have:

• An implementation of MPI. Parallel machines now come with one provided by the vendors (e.g., IBM SP, Cray T3E, and SGI machines); check with your support staff. We recommend using the vendor-provided implementation of MPI if it has been installed, since this usually provides better performance than the freeware versions. Otherwise, we recommend MPICH, which is available at http://www.mcs.anl.gov/mpi/mpich. If MPI is not yet installed on your system, retrieve a version of MPI, install it, and run the example programs before proceeding further. 

  For users who are interested ONLY in running PETSc sequentially, we have included a stripped-down version of MPI that allows PETSc to be compiled without installing an external version of MPI. This setup allows the user to run ONLY uniprocessor PETSc  programs, not parallel ones.

• A copy of the BLAS and LAPACK. Many machines provide math libraries that contain BLAS or LAPACK. For example, the DEC alpha provides DXML, and the IBM rs6000 provides ESSL. (Note that LAPACK must still be installed even if linking with ESSL, since ESSL does not contain all  LAPACK routines.)  Check with your support staff. The BLAS library on some machines may be found as /usr/lib/libblas.a. If these libraries are not already installed on the target architecture, they can be obtained from  ftp://info.mcs.anl.gov/pub/petsc/blas_lapack.tar.gz. The makefiles with this are intentionally simple, since they require editing to fit the particular machine. We recommend using the vendor-provided BLAS when possible.

Optional Packages:

PETSc provides an interface to several software packages.. These packages are not developed, maintained, or supported by the PETSc team; we merely provide an interface to them. To use any of these optional packages, obtain the following prior to installing PETSc:

• BlockSolve95, a software package of parallel ICC(0) and ILU(0) preconditioners from http://www.mcs.anl.gov/blocksolve95/index.html.

Before compiling BlockSolve95 make sure that the BlockSolve file src/makefile has at the bottom
CFLAGS = ${BS_INCLUDE} ${MPI_INCLUDE} -DMLOG
The -DMLOG is extremely important.

• BlockSolve95 does not support complex numbers; hence, it cannot be used with any of the complex number versions of PETSc.
• If you want to use BlockSolve with BOPT=g_c++ or BOPT=O_c++ on the SGI machines, PETSC_ARCH=[IRIX,IRIX64], you must first fix one of the BlockSolve95 include files. Edit the file BlockSolve95/include/BSdepend.h and remove the line
  • extern void srand(int);
• If you want to use BlockSolve95 with BOPT=g_c++ or BOPT=O_c++ on PETSC_ARCH=freebsd or PETSC_ARCH=linux, or using the Gnu g++ compiler version 2.7.2 or higher on the sun4, you must edit the above file and remove the line
  • extern int exit()

• ParMETIS, a parallel graph partitioner, from http://www.cs.umn.edu/~metis .
• The ALICE Memory Snooper (AMS), from http://www.mcs.anl.gov/ams.   This package allows one to monitor (and change) variables in running PETSc programs (or, more generally, any programs using MPI).  PETSc objects, such as matrices and solvers, can also be monitored directly from external programs.  See the manual page for ViewerAMSOpen() for details on AMS usage in PETSc.
• SPAI 3.0 (Sparse approximate inverse code by Steve Barnard)   It may be obtained by sending email to barnard@nas.nasa.gov

Installing PETSc:

• The PETSc distribution can be unbundled with
     gunzip -c petsc.tar.gz | tar xof -
  By default, this will create a directory called petsc-2.0.24 and unpack the software there.
• Refer to http://www.mcs.anl.gov/petsc/petsc-patches.html for fixes for the latest PETSc release.
• Set the environmental variable PETSC_DIR to the full path of the PETSc home directory, for example,
     setenv PETSC_DIR /home/username/petsc-2.0.24
• Set the environmental variable PETSC_ARCH, which indicates the architecture on which PETSc will be configured. For example, use
     setenv PETSC_ARCH  `$PETSC_DIR/bin/petscarch`
• Edit the file ${PETSC_DIR}/bmake/${PETSC_ARCH}/base.site to indicate the local installation of MPI, LAPACK, BLAS, X-windows, and the optional software packages.

  Note: If installing ONLY a uniprocessor version of PETSc, then installation of an MPI implementation is not required. Instead, the following MPI locations can be used in the ${PETSC_DIR}/bmake/${PETSC_ARCH}/base.site file:
     MPI_LIB = ${PETSC_DIR}/lib/lib${BOPT}/${PETSC_ARCH}/libmpiuni.a
     MPI_INCLUDE = -I${PETSC_DIR}/src/sys/src/mpiuni
     MPIRUN = ${PETSC_DIR}/src/sys/src/mpiuni/mpirun

• It may also be necessary to edit the following file   ${PETSC_DIR}/bmake/${PETSC_ARCH}/base_variables to change the names of the C, C++, or Fortran compilers from their defaults:

• Solaris using the GNU compilers: use PETSC_ARCH=solaris_gnu
• IBM rs6000 using the GNU compilers: use PETSC_ARCH=rs6000_gnu
• Sun4 machines: if using the Sun ANSI-C compiler, these files must be edited accordingly.
• CRAY t3d: make sure the enviornmental variable TARGET is set to cray-t3d

• In the PETSc home directory, type
     make BOPT=g all >& make_log
  to build a debugging version of the PETSc or
     make BOPT=O all >& make_log
  to build optimized version of the PETSc libraries. The flag BOPT determines what type of libraries are built (i.e., specifies compiler options). The other alternatives are BOPT=[g_c++,O_c++] for C++ versions and BOPT=[g_complex,O_complex] for complex numbers versions.

  On HP-UX machines we highly recommend using Gnu make (which can be installed if not already available) rather than the native make. To switch to using gnumake in the PETSc makefiles, define OMAKE to be the path of gnumake in the file ${PETSC_DIR}/bmake/hpux/base.

• Check make_log to determine whether any errors have occured during the installation process. Try to fix any errors, and let us know what they were. But first see the files troubleshooting.html , http://www.mcs.anl.gov/petsc/petsc-patches.html and docs/bugreporting.html for help with common problems.
• To install the SPAI interface
  cd src/contrib/pc/spai/src; make BOPT=g

Test programs:

• If the installation went smoothly, then try running some test examples with the command
     make BOPT=g testexamples >& examples_log 
• If only the uniprocessor version of PETSc has been installed (i.e., MPI has not been installed), then use the following command to run only sequential examples:
     make BOPT=g testexamples_uni >& examples_log
• Examine the file examples_log for any obvious errors or problems.
• The examples can be manually built and run one at a time by changing to the appropriate directory (for instance, ${PETSC_DIR}/src/sles/examples/tutorials) and running commands such as
     make BOPT=g ex1
     make runex1 (or, for example, mpirun ex1)
  This alternative may be preferable if "make BOPT=g testexamples" fails for some reason.
• The automatic tests may not work on systems that use a queue and special commands to run parallel jobs.  Instead, the user can compile and run the examples manually as discussed above.
• To test the graphics examples, move to ${PETSC_DIR}/src/sys/src/draw/examples/tests; then make and run the examples manually. These examples will open an X window and draw some graphics.

Fortran Users:

The PETSc Fortran libraries are built automatically during the installation outlined above. Before testing the fortran examples, please make sure that the c-version of the examples work correctly.

• To compile and test the Fortran examples, use the command
     make BOPT=g testfortran >& fortran_log

PETSc Fortran programs can use the suffix .F rather than the traditional suffix .f, so that the PETSc header files can be easily included in Fortran programs. See the Fortran chapter within the users manual for additional details regarding the Fortran interface. In order to use the suffix .f instead of .F, the user must edit the file ${PETSC_DIR}/include/foldinclude/petsc.h to hardwire the path for the local MPI include file. See the chapter 'PETSc Fortran Users' in the users manual for more information.

Multiple Installations:

When building PETSc for two or more machine types that share a common filesystem, for example sun4 and hpux, multiple PETSc directory trees are NOT needed. Only a single PETSc directory can (and should) be used; PETSc automatically places the libraries for each machine in a different location. In particular, the libraries for a given BOPT and PETSC_ARCH are installed in the directory, ${PETSC_DIR}/lib/lib${BOPT}/${PETSC_ARCH}.

Shared Libraries:

PETSc supports the use of shared libraries for the machines sun4, solaris, alpha, IRIX, IRIX64, freebsd, and linux to enable faster linking and smaller executables.  These libraries are built automatically during installation. In addition, PETSc now defaults to using these libraries as dynamic libraries on these machines. For most users this does not matter.

Installation under Windows NT/95/98

We currently distribute PETSc libraries to work in uniprocessor mode, under Windows NT/95/98 using Microsoft Visual C++ 5.0 compilers.

See http://www.mcs.anl.gov/petsc/petsc-patches.html and troubleshooting.html for help with problems.

Installing PETSc:

• Run petsc.exe and one or either of petsc-c.exe, petsc-c++.exe or petsc-complex.exe,and they will create a directory
  c:\petsc-2.0.24
  containing all the source code, documentation and (C and or C++) libraries.
• Add
    set PETSC_DIR=c:\petsc-2.0.24
    set PETSC_ARCH=win32
  to your autoexec.bat (for Windows 95) or
  
  Settings->Control Panel->System->Environment Tab->User Variables (for Windows NT)

Running Examples from a Shell:

• Verify that nmake and cl (the Microsoft make utility and compiler) run from the command line;  if  they do not, then set them up.
• Change to an example directory:  cd c:\petsc-2.0.24\src\is\examples\tutorials
• Compile an example program: nmake BOPT=g ex1 (note it is nmake not make)
• Run this example program:  ex1
• Try other examples in the directories src\*\examples\tutorials and src\*\*\examples\tests

Running Examples from Microsoft Visual Studio: (using a Project)

Two project files, one for C libraries, and the other for the C++ libraries, are provided with the PETSc distribution.They are located in c:\petsc-2.0.24\projects\[c,c++]\sles\ex2 directory. To build and run the example file using the project provided:

• Make sure that the environment variables PETSC_ARCH and PETSC_DIR are set correctly and are accessable to the Visual Studio.
• Load the project file ex2.dsw into Microsoft Visual Studio.
• To compile the example, use  Menu ->Build -> Build ex2.exe (F7)
• To run the example, use Menu -> Build -> Execute ex2.exe   (Ctrl-F5)

To create a project for your files:

• From c:\petsc-2.0.24\projects\[c,c++]\sles\ex2 Copy the files - ex2.dsw and ex2.dsp,   to the location of your project directory.
• Rename the files to your desires project name - say - my_prj.dsw and my_prj.dsp
• Load the new project file my_prj.dsw into Microsoft Visual Studio.
• From file view stub, remove the source file ex2.[c,cpp]
• From Menu Project -> add to project -> files, add the user source files to the project.

Program Arguments: can be set in Developer Studio from the Menu Project -> settings ->Debug -> program arguments.

Currently we only provide project examples using console applications; developing a windows application requires creating a new project.

Fortran Users:

If you have the Digitial Visual Fortran compiler (with the latest patches), you can compile PETSc Fortran code

• Verify that nmake and df (the Microsoft make utility and fortran compiler ) run from the command line;  if  they do not, then set them up.
• Change to an example directory:  cd c:\petsc-2.0.24\src\is\examples\tutorials
• Compile an example program: nmake BOPT=g ex1f
• Run this example program: ex1f

Running Fortran examples from Developer Studio:

Currently this is not possible, as we need to use the build targes provided in the PETSc base files.

Rebuilding Libraries

The Windows version of PETSc comes with the C and C++ version libraries pre-built. If, for some reason you need to rebuild the libraries, you can do the following:

• Verify that  nmake and cl (the Microsoft make utility and compiler ) run from the command line; if  they do not, then set them up.
• If you made changes in single directories, then run
     nmake BOPT=g or
     nmake BOPT=O from those directories
• To rebuild all the libraries (debugging version), from the main PETSc directory run
     nmake BOPT=g all  from the main PETSc directory (or use the O option).

Using Microsoft Visual C/C++ Compiler from the Command Line:

If the commands nmake and vc do not work from the DOS command line, but you do have Microsoft Developer Studio installed, you can either

• run the script Program Files\DevStudio\VC\bin\vcvars32.bat or
• edit autoexec.bat (Windows 95) and add the variables
  set LIB=C:\Program Files\DevStudio\DF\LIB;C:\Program Files\DevStudio\VC\LIB;%LIB%
  set INCLUDE=C:\Program Files\DevStudio\DF\INCLUDE;C:\Program          Files\DevStudio\VC\INCLUDE;%INCLUDE%
  set PATH="C:\Program Files\DevStudio\SharedIDE\BIN";"C:\Program Files\DevStudio\DF\BIN";"C:\Program Files\DevStudio\VC\BIN";"%PATH%"

Complex version of PETSc libraries

To use the complex version of PETSc libraries, you need Digital Visual Fortran compiler along with Microsoft's Visual C++ compiler. Before you can compile the C example files, edit the file bmake/win32/base and chnage the varaible CXX_CLINKER to use the fortran compiler instead of the C++ compiler.

Using MPI on Windows NT/95/98

The default PETSc libraries, that come with the distibution work in uniprocessor mode only. You can use PETSc with either of the following implementations of MPI

• WMPI - an implementation of MPI form Coimbra University, Portugal. This implementation of MPI works on Windows NT as well as Windows 95/98. To make sure that WMPI works from both C and fortran PETSc examples, download the version of WMPI compiled for PETSc
• HPVM - an implementation of MPI from University of Illinois at Urbana-Champaign and University of California, San Diego. This implementation of MPI works on Windows NT only.

To install PETSc with MPI, you would have to do the following:

• Edit the file bmake/win32/base.site and change the variables MPI_INCLUDE and MPI_LIB to point to the installation location of MPI.
• Edit the file bmake/win32/base and make sure that the compile flags specified in COPTFLAGS and FOPTFLAGS match the compile flags used to build MPI.
• Now rebuild the PETSc libraries.

Installing PETSc Using gcc/g77:

PETSc can be installed on Windows NT/95 using gnu-win32  tools from  Cygnus Solutions, (which contains bash and make) and EGCS version of gcc, g++, g77 compilers. Currently we use gnu-win32 version beta-20 and EGCS compilers for gnu-win32 beta 20.

• Get the unix distribution of petsc. i.e, petsc-2.0.24.tar.gz.
• To build blas, lapack, use g77.
• To build PETSc, use the following:
     bash  (use the bash shell which is part of gnu-win32)
     PETSC_ARCH=win32_gnu
     PETSC_DIR=c:/petsc-2.0.24
• To build the libraries (debugging version), run
     make BOPT=g all