Graph partitioning is a useful abstraction for load balancing. The main idea is to represent the computational application as a weighted graph. The nodes or vertices in the graph correspond to objects in Zoltan. Each object may have a weight that normally represents the amount of computation. The edges or arcs in the graph usually correspond to communication costs. In graph partitioning, the problem is to find a partitioning of the graph (that is, each vertex is assigned to one out of k possible sets called partitions) that minimizes the cut size (weight) subject to the partitions having approximately equal size (weight). In repartitioning, it is assumed that a partitioning already exists. The problem is to find a good partitioning that is also "similar" in some sense to the existing partitioning. This keeps the migration cost low. All the problems described above are NP-hard so no efficient exact algorithm is known. We remark that in Zoltan 1.*, the number of partitions is always the same as the number of MPI processes (which is normally equal to the number of processors).
We give only a brief summary of the various ParMETIS methods here; for more details see the ParMETIS documentation. The methods fall into three categories:
Zoltan is currently compatible with ParMETIS versions 3.1 and 2.0. There is no guarantee that Zoltan will work correctly if you have a different version of ParMETIS on your computer. (ParMETIS 3.0 will work with Zoltan in most cases, but is not officially supported. ParMETIS 3.1 is highly recommended. The 2.0 version will soon become obsolete and may not be supported in future Zoltan versions.) The ParMETIS source code can be obtained from the ParMETIS home page. As a courtesy service, a recent, compatible version of the ParMETIS source code is distributed with Zoltan. However, ParMETIS is a completely separate library. If you do not wish to install ParMETIS, it is possible to compile Zoltan without any references to ParMETIS (when you 'make' Zoltan, comment out the PARMETIS_LIBPATH variable in the configuration file Utilities/Config/Config.<platform>).
Note that Zoltan ignores the imbalance tolerance parameter IMBALANCE_TOL when ParMETIS 2.0 is used (the default value 1.05 is used), while IMBALANCE_TOL works correctly with ParMETIS 3.0. Zoltan supports the multiconstraint feature of ParMETIS 3 through multiple object weights (see OBJ_WEIGHT_DIM).
The graph given to Zoltan/ParMETIS must be symmetric. Any self edges
(loops) will be ignored. Multiple edges between a pair of vertices is not
allowed. All weights must be non-negative. The graph does not have to be
connected.
Method String: | GRAPH or PARMETIS |
Parameters: | |
PARMETIS_METHOD | The ParMETIS method to be used; currently nine are available.
PartKway - multilevel Kernighan-Lin partitioning PartGeom - space filling curves (coordinate based) PartGeomKway - hybrid method based on PartKway and PartGeom (needs both graph data and coordinates) AdaptiveRepart - adaptive repartitioning (only in ParMETIS 3.0 and higher) RepartLDiffusion - diffusion algorithm (local) RepartGDiffusion - diffusion algorithm (global) RepartRemap - multilevel partioning with remap seeking to minimize migration cost RepartMLRemap - similar to RepartRemap but with additional multilevel refinement RefineKway - refine the current partitioning (balance) |
The method names are case insensitive. | |
PARMETIS_OUTPUT_LEVEL | Amount of output the load-balancing algorithm should produce.
0 = no output, 1 = print timing info. Turning on more bits displays more information (for example, 3=1+2, 5=1+4, 7=1+2+4). |
PARMETIS_COARSE_ALG | Coarse algorithm for PartKway. 1 = serial, 2 = parallel. (ParMETIS 2 only) |
PARMETIS_SEED | Random seed for ParMETIS. |
PARMETIS_ITR | Ratio of interprocessor communication time to redistribution time. A high value will emphasize reducing the edge cut, while a small value will try to keep the change in the new partition (distribution) small. This parameter is only used by AdaptiveRepart. A value of between 100 and 1000 is good for most problems. |
USE_OBJ_SIZE | Use (or not use) the available information about object sizes to estimate migration cost. This parameter is currently only relevant for AdaptiveRepart. |
CHECK_GRAPH | Level of error checking for graph input: 0 = no checking, 1 = on-processor checking, 2 = full checking. (CHECK_GRAPH==2 is very slow and should be used only during debugging). |
SCATTER_GRAPH | Scatter graph data by distributing contiguous chunks of objects (vertices) of roughly equal size to each processor before calling the partitioner. 0 = don't scatter; 1 = scatter only if all objects are on a single processor; 2 = scatter if at least one processor owns no objects (recommended to avoid a bug in ParMETIS 2.0); 3 = always scatter. |
Default values: | |
PARMETIS_METHOD = RepartGDiffusion | |
PARMETIS_OUTPUT_LEVEL = 0 | |
PARMETIS_COARSE_ALG = 2 | |
PARMETIS_SEED = 15 | |
PARMETIS_ITR = 100 | |
USE_OBJ_SIZE = 1 | |
CHECK_GRAPH = 1 | |
SCATTER_GRAPH = 1 | |
Required Query Functions: | |
For all submethods: | ZOLTAN_NUM_OBJ_FN |
ZOLTAN_OBJ_LIST_FN or ZOLTAN_FIRST_OBJ_FN/ZOLTAN_NEXT_OBJ_FN pair | |
Only PartGeom & PartGeomKway: | ZOLTAN_NUM_GEOM_FN |
ZOLTAN_GEOM_MULTI_FN or ZOLTAN_GEOM_FN | |
All but PartGeom: |
ZOLTAN_NUM_EDGES_MULTI_FN or
ZOLTAN_NUM_EDGES_FN
ZOLTAN_EDGE_LIST_MULTI_FN or ZOLTAN_EDGE_LIST_FN |