The RESIDENT Clause, Directive, and Construct

 

The purpose of the RESIDENT clause is to promise that data accessed by a computation are mapped to active processors. That is, RESIDENT asserts that certain references (or all references) in its scope are stored on the active processor set. The compiler can use this information to avoid generating communication or to simplify array address calculations. Note that whether a given data element is resident depends on two facts:

• Where the data is stored (i.e., DISTRIBUTE and ALIGN attributes for the object)
• Where the computation is executed (i.e., its active set, as specified by an ON directive)

For these reasons, the RESIDENT clause is added to the ON directive, which is usually the earliest point in the program where the needed facts might be available. The RESIDENT clause can also appear as a stand-alone directive; this is useful when the locality information is not true for an entire ON region. Note that changing the ON directive may invalidate some RESIDENT clauses, or may make more RESIDENT clauses true.

H910 resident-clause	is RESIDENT resident-stuff
H911 resident-stuff	is [ ( res-object-list ) ]
H912 resident-directive	is RESIDENT resident-stuff
H913 resident-construct	is directive-origin block-resident-directive block directive-origin end-resident-directive
H914 block-resident-directive	is RESIDENT resident-stuff BEGIN
H915 end-resident-directive	is END RESIDENT
H915 res-object	is object

A resident-directive is a kind of executable-directive. Similarly, a resident-construct is a kind of executable-construct.

Any top-level objects in the RESIDENT clause must be explicitly mapped. Similarly, the RESIDENT clause must appear at a point in the program with a declared active processor set (i.e., inside an ON block). Otherwise, the assertion (see below) is a statement about how the compiler works, not about the program.

Advice to implementors.RESIDENT removes the need for inactive processors to participate in communication into/out of an ON clause. (End of advice to implementors.)

The RESIDENT directive is an assertion to the compiler that certain object references made within the ON are stored on the active processors if the computation is performed by the specified active processor set. The scope of the assertion is the next Fortran statement if the resident-directive form is used and the enclosed block of code if the resident-construct form is used. If RESIDENT appears as a clause in an ON directive, then the ON and RESIDENT apply to the same statements.

RESIDENT(var) means the lexical expression var, when encountered in the execution of statements in the scope of the RESIDENT directive, accesses only data resident on the set of active processors. (That is, the set of processors named by the innermost available ON directive.) If var is accessed by the statement (e.g., it appears on the right-hand side of an assignment statement, or in the evaluation of a conditional expression), then at least one copy of the object and any subobject of the object must be mapped to the active processor set. If var is assigned to by the statement (e.g., it appears on the left hand side of an assignment statement, or in the variable list of a READ statement), or in any other context that may cause its value to change) then all copies of the variable and all subobjects of the variable must reside in the active processor set. The application of RESIDENT to CALL statements and function invocations introduces some complexity into this interpretation; these issues will be dealt with in Section 9.3.2.

Note that RESIDENT is always an assertion relative to the surrounding ON directive. Therefore, if the compiler does not implement the ON directive then it must be careful in interpreting RESIDENT. Similarly, if the compiler overrules the programmer-specified ALIGN and DISTRIBUTE directives, then it may not rely on the RESIDENT clause in general.

Finally, NEW variables are not considered by any nested RESIDENT directives, as detailed below.

Rationale.The different treatment of variable reads and writes is due to the implementation requirements. If a variable's value is read (but not written), then it can be taken from any consistent copy. Therefore, RESIDENT only asserts that one of those copies is available. Conversely, all copies of a replicated variable must be consistent, so RESIDENT asserts that all copies are available if it is updated.

The RESIDENT assertion is always relative to the declared data mappings and ON clauses because both pieces of information are necessary to determine the locality of data references. Data mapping determines where the data is stored, while ON clauses determine where they are used; in essence they determine the endpoints of a data path. RESIDENT itself says that the path length is very short; obviously, one cannot measure a path without knowing both endpoints. (End of rationale.)

Consider the following:

!HPF$ ON HOME(Z(I)), RESIDENT(X,Y,RECORD(I))
      X(I) = Y(I+1) + RECORD(I)%FIELD1 + RECORD(I+1)%FIELD2

The following facts are asserted by the directive:

• Z(I) would be local if it appeared, due to its use in the HOME directive.
• All copies of X(I) are stored on processors that also store a copy of Z(I), due to the RESIDENT clause. This may be true because X and Z have the same mapping, or because Z is replicated over a set of processors that contains the set of processors that store X(I).
• At least one copy of Y(I+1) is on the same processor as Z(I), due to the RESIDENT clause. This may be true because Y is replicated on all processors, because Z(I) and Y(I+1) are the only elements of their arrays that are mapped to the same processor, or because the directive

  !HPF$ ALIGN Y(J) WITH Z(J-1)
  

  appears elsewhere in the program. (Other situations also make the RESIDENT assertion true.)

• At least one copy of all subobjects of RECORD(I) is mapped on the same processor as Z(I). In particular, the reference RECORD(I)%FIELD1 (i.e., a subobject consisting of one component) can be accessed locally. The situations in which this is true are similar to those for X(I). No information is available in this example regarding RECORD(I+1)%FIELD2.

If there is no res-object-list, then all references to all variables referenced during execution of the RESIDENT directive's body except those declared NEW in a surrounding ON directive are local in the sense described above. That is, for every usage of any variable's value, at least one copy of the variable will be mapped to the ON processor set. Likewise, for every operation that assigns to a variable, all copies of that variable are mapped to the ON processor set. References and assignments to NEW variables are always considered resident. If there are no function or subroutine invocations, this is syntactic sugar for listing all variable references within the directive's scope. (See Section 9.3.2 for a discussion of RESIDENT clauses applied to subprogram calls.) It might well have been named the ALL_RESIDENT clause; the present form, however, does not add yet another keyword to the directive sublanguage.

Note that if the active set includes more than one processor, then RESIDENT only asserts that the variables are stored on one of the processors. For example, if a statement is executed on a section of the processors arrangement, then communication within that section may be needed for some variables in the RESIDENT clause. Communication with processors outside of the section will not be needed for those variables, however.

Rationale.The alternative to this interpretation would be that any variable named in the RESIDENT clause would be local to all processors, i.e., replicated. While that certainly allows more extensive optimizations, it is a less common case. In addition, it does not seem to capture the intent of ON directives applied to CALL statements or compound statements. For example,

!HPF$ PROCESSORS PROCS(MP,MP)
!HPF$ DISTRIBUTE X(BLOCK,BLOCK) ONTO PROCS
!HPF$ ON HOME(PROCS(1,1:MP)), RESIDENT(X(K,1:N) )
      CALL FOO( X(K,1:N) )

would presumably call FOO on a row of the processors arrangement, passing elements of X in place. This is what the current definition does; if RESIDENT meant ``resident on every processor'', the call would force X to be replicated. (End of rationale.)

It is not correct to assert that an unmapped object, including of necessity any sequential object, is mapped exclusively to the active processors, unless the programmer knows that all of the processors are active. Thus, when a proper subset of processors is active, no such object can occur in a res-object-list or in the scope of a RESIDENT directive with no res-object-list.

The RESIDENT directive is similar to the INDEPENDENT directive, in that if it is correct it does not change the meaning of the program. If the RESIDENT clause is incorrect, the program is not standard-conforming (and is thus undefined). Like the INDEPENDENT directive, the compiler may use the information in the RESIDENT clause, or ignore it if it is insufficient for the compiler's purposes. If the compiler can detect that the RESIDENT clause is incorrect (i.e., that a RESIDENT variable is definitely nonlocal), it is justified in producing a warning. Unlike the INDEPENDENT directive, however, the truth of the RESIDENT clause depends on the mapping of computations (specified with the ON clause) and the mapping of data (specified with DISTRIBUTE and ALIGN clauses); if the compiler overrides either of these, then it may not be able to use information in the RESIDENT directive.

Rationale.Knowing that a reference is local is valuable information for the optimizer. It is in keeping with the spirit of HPF to phrase this as an assertion of fact, which the compiler can use as it pleases. Expressing it as advice to the compiler seems to have disadvantages. Some possible ways this advice could be phrased, and the counter-arguments, are
• ``Don't generate communication for this reference'' has great potential for changing the meaning of the program. Some programmers want this capability, but it violates the ``correct directives should not change the meaning of a program'' principle of HPF. Also, once communication is ``turned off'' for a reference, it's not clear how to turn it back on.
• ``Generate communication for this reference'' is not a useful directive, since the compiler has to do this anyway.
• ``Generate communication for this reference, and place it here'' is useful, since it can override the default placement by the compiler. It still has potential for changing program meaning. It also has the potential to create programs as complex as message-passing, as programmers try to move communication out of loops.
(End of rationale.)

 
• Examples of RESIDENT Clauses
• RESIDENT Directives Applied to Procedure Reference