Abstract Data Model

The term "mesh" refers to an abstraction in the data model; it does not imply a serial or parallel distribution.
The term "partition" refers to an assignment of a set of entities to subsets; like a "mesh," it does not imply a serial or parallel implementation.
An application may use one or more meshes.
Partitions can create subsets of entities from one or more meshes.
Meshes can be subdivided by one or more partitions.
Partitions contain parts. Parts contain the subsets of entities in the partition.

Parallelism

A "process" can be thought of as an MPI process. The number of processes can be considered to be the result of MPI_Comm_size. The rank of a process can be thought of as the result of MPI_Comm_rank. We will think in terms of processes rather than processors. Initial implementations of the parallel interface will likely use MPI terminology directly; future implementations may accommodate other communication paradigms and libraries.
Partitions have communicators associated with them. These communicators can be thought of as MPI communicators.
"Global" operations are operations performed with respect to a partition's communicator.
"Local" operations are operations performed with respect to a part or a mesh instance within a process.
Part A "neighbors" Part B if Part A has copies of entities owned by Part B and/or if Part B has copies of entities owned by Part A.

Interfaces

Each process has one or more "mesh instances." A mesh instance can be thought of as a mesh database. An implementation should support the existence of more than one mesh instance per process (e.g., it should always associate mesh data with a mesh instance). However, we expect applications would most often use only one mesh instance per process.
There is one root set per mesh instance.
Each process may have one or more partition handles.
A partition assigns entities from one mesh instance to parts.
Entities in a mesh instance can be partitioned by one or more partitions. Mesh instances know which partitions they contain.
Parts are uniquely identified globally by part IDs of type iMeshP_Part. Local parts can also be accessed by part handles that provide more direct access to a part. Functions accepting part handles operate correctly on only local parts (parts on the calling process); they will return an error for remote (off-process) parts.
Generation and management of global IDs for entities is not included in the iMeshP interface. It can be provided as a service above the iMeshP interface. Uniqueness of global IDs is managed at the partition level.

Using Parts

Each part is wholly contained within a process.
A process may have zero, one or multiple parts.
For each entity that is copied onto remote parts, the owning part knows both the remote part ID and remote entity handle of all copies.
All parts with copies of a boundary entity know the remote part ID and remote entity handle of all copies of the entity.
All parts with copies of any entity know the part ID and entity handle corresponding to the owner of the entity.
Functions that return entity information for a part, set or mesh instance return the information for all entities (including copies and ghosts) in that part, set or mesh instance. Applications can check whether an entity is owned or a ghost using iMeshP_isEntOwner or iMeshP_getEntStatus.
Many iMesh functions that accept an iBase_EntitySetHandle are also useful in the context of a iMeshP_PartHandle. These functions are reinterpreted so that they can accept either an iBase_EntitySetHandle or an iMeshP_PartHandle.
In particular, entities are added to and removed from local parts via the same functions that are used to manipulate entity sets. That is, given a mesh instance, an entity handle, and a part handle, the entity is added to or removed from the part via calls to the following functions with the part handle passed as the entity set handle:
- Add entity to part --> iMesh_addEntToSet
- Remove entity from part --> iMesh_rmvEntFromSet
- Add array of entities to part --> iMesh_addEntArrToSet
- Remove array of entities from part --> iMesh_rmvEntArrFromSet

Communication

Each function description includes its communication requirements. The options are described here:
- COMMUNICATION: Collective -- the function must be called by all processes in the partition's communicator. (These functions have the suffix "All" to indicate collective communication is done.)
- COMMUNICATION: Point-to-Point -- communication is used, but the communication is from one process to only one other process. The receiving process must issue an appropriate receive call to receive the message.
- COMMUNICATION: None -- the function does not use communication; only local operations are performed.
- COMMUNICATION: None++ -- no communication is done; the values are precomputed by iMeshP_syncPartitionAll or iMeshP_syncMeshAll.
Non-blocking calls for off-processor mesh-modification return a request that indicates whether or not the operation has completed. The request is more than an MPI request; it encapsulates both the MPI information and the mesh operations that were requested. If non-blocking calls are used, appropriate calls to iMeshP "wait" or "poll" functions must be used to handle and satisfy requests.