May 11-12, 1995, Denver Colorado
Record of Action: Mary Zosel
Executive Summary
Thirty people representing 27 organizations attended the first of the new round of High Performance Fortran Forum Meetings. The meeting began with usual administrative issues. Proposal procedures (2 readings) and voting rules are the same as previous HPFF meetings. The objectives of the group are:
The status of the vendor implementation list is unchanged since January.
Dates and tentative locations selected for the meetings through January 96: July 26-28, Denver; Sept 20-22, Dallas area; Nov 1-3, Dallas area; January 9-12 Houston (in conjunction with annual HPF implementation and usage experience meeting) Further meetings in 96, are to be determined, as needed.
The subject matter to be addressed was divided into three subgroups:
The charter of the groups is to produce appropriate language proposals, and to resolve CCI issues. Details of the topics to be addressed by these subgroups are given in the full minutes following.
Two documents are expected to emerge from the current meetings: an updated HPF1.1. with additional corrections, and a new HPF 2.0 document reflecting new language features that are adopted by the group.
Afternoon and evening subgroup meetings were held to plan subgroup activities and address CCI items, with reports on these activities on Friday AM.
For the July meeting, we expect to prepare comments for the US public review of the Fortran 95 proposal and also hope to have initial presentations for several issues including asyncronous I/O and language interoperability.
End of Executive Summary
Detailed Record of Action
Ken Kennedy called the meeting to order at 8:40 May 11. Introductions were held and the initial count of organizations represented was 23. (More people arrived later, bring the final number to 30 people representing 27 organizations.)
Voting rules were reviewed: on formal votes, such as 2nd reading of a proposal, there is 1 vote per organization. On straw-poll votes, all present can vote. To be eligible to vote, an organization must have been present at one of the previous two meetings. This rule will take effect at the September meeting. Continuity is important, so it is hoped that the same individual from each organization can attend on a regular basis.
Proposal processing will be as before, with a first reading to get a sense of approval of direction from the entire committee, followed by a second reading before the proposal is formally adopted.
There was a discussion of what documents will be produced and appropriate names. Two documents will be produced. An update of version 1.1 with corrections, based on the CCI work from last year will be called HPF 1.2. If additional corrections are identified during this year, HPF 1.3 might be produced. For the draft versions of the HPF 2 document, Ken suggests that we start with HPF 1.5.
CCI items submitted during the HPF 2 meetings will be handled by the subgroups. It will be up to subgroup chairs to decide whether or not an issue needs to come to the full group for discussion and vote.
The vendor list was reviewed. KAI was taken off of the announced product list and there is a question about the status of ACSET.
Meeting schedule for the HPF 2 meetings was reviewed. Dates of July 26-28 and Sept. 20-22 have already been selected. Further dates proposed are Nov. 1-3 and Jan 9-12, including the annual 1.5 day meeting. Another meeting around the end of February or early March was discussed. This and other 1996 meetings will be scheduled as needed to complete the document. Since the November meeting follows Halloween closely, we are expecting some members to show up as masked elsewheres.
There was a general feeling that the meetings after July should be in another location. A straw poll gave slight preference for Dallas over Chicago (11 to 9). For now, the September and November meetings will be planned for Dallas. A hotel closer to DFW - perhaps in Arlington, will be investigated. The January meeting will be in Houston.
Jerry Wagoner gave an update of the status of Fortran 95. The process started out as mostly CCI and minor things, but HPF influenced adding most of the HPF syntax. The language now is F90 plus forall, partial nesting of forall and where statements, masked elsewhere, pure procedures, elemental procedures, pure procedures in specification expressions, revised minloc and maxloc, pointer initialization,default initialization of derived type objects, increased compatibility with IEEE arithmetic, and a CPU_TIME intrinsic subroutine. Five features are deleted: arithmetic IF, real DO loop index variables, ASSIGN statements, assigned GO statements, and PAUSE statements.
Two important issues, exception handling, especially for floating point, and interoperability between languages are not addressed in F95. There was a strong request for the floating point exception handling, but the full exception handling processing got too big and was removed. Public review may cause it to be restored. Ken pointed out that many of the algorithms that need floating point exceptions are the kind inspired by Jim Demmel - and are important for parallel computing so this might become an HPF issue. The general idea is that a fast algorithm is applied and only if a floating point exception arises does a slower method get invoked. Jerry can give a tutorial on this at the next meeting. About interoperability, there is some controversy about the proper mechanism. There is a proposal to add a bunch of directives. WG5 has formed two working groups to study these areas and make proposals. Jamie Shiers of CERN is the contact for interoperability and Wolfgang Walter of the University of Dresden is the contact for floating point exceptions.
The expected schedule for Fortran 95, as of May, is:
April 95 technical compete;
end of May 95 document available
~June-Aug95 review and ISO CD ballot
Nov. 95 revision based on CD ballot comments
Jan 96 revised document available
Feb-July 96 ISO DIS ballot
Aug 96 published as an ISO standard (replacing Fortran 90)
Jerry suggest that HPF can help with this process. Is the nesting of forall and where correct? And can HPF generate some input for exceptions and interoperability. At the July meeting we will try to get a letter from HPFF with comments. Jerry also said that he has written a letter to ANSI X3 Secretariat stating that we expect to exchange text freely.
After a break, the group reconvened with some experience reports about implementations of features we might be considering. Joel Saltz reported on research using various approaches to irregular partitioning. Barbara Chapman summarized the Vienna Fortran features similar to what we are considering for HPF. She observed that reductions occur frequently in loops so optimized code is important - and many cases can be recognized automatically. In their work with the ON clause, if there is only one left-hand-side array, the owner computes rule is generally superior, but irregular distributions and the regular distribution of (block,cyclic) produced poor performance. Another issue identified was the owner function for multi-dimensional irregular distributions.She raised the question of how frequently there is actually a need for multi-dimensional irregular structures. The generalized block distribution allows formuch smaller amount of memory. The block sizes are irregular - but the about of memory. The blocks sizes are irregular - but the number of neighbors # of neighbors fixed. Rob Schreiber next reviewed the issues for mapping components of derived types and scalars. Jaspal Subhlok discussed parallel tasks in HPF where data and computation are mapped together, but without explicit communication or shared space between tasks. Finally, Joel talked about a generalized reduction operations such as reduce_append to add particles to new list. A functions like this can result in a 10-times speedup for this operation that is common in PIC codes.
The next order of business was formation of subgroups. Ken presented a proposal for three subgroups - and people signed up for groups of their choice. Following is the breakdown of the subgroup structure, topics, membership, and chair persons.
Group C : "control-flow-related issues"
asynchronous I/O
ON for do loop iteration
reductions for intrinsic and user-defined ops
restricted parallel sections (no communication)
general issues related to chap. 4, 5, 6 of HPF 1.1. doc
F95 feedback about nested structures and fp exceptions
related CCI questionsRob Schreiber chair: Chuck Koelbel, Ian Foster, Jaspal Subhlok, Joel Williamson, Joel Saltz, J. Ramanujam, Larry Meadows, Ken Kennedy.
Group D: "Distribution-related issues"
map arrays for irregular mapping
processor sections as target in distribute
mapping assertions for reuse of communication. patterns
mapping scalars to processors
mapping derived type components
mapped I/O
general issues related to chap 3 & 7 of HPF 1.1 doc
related CCI questionsGuy Steele chair, Carl Offner, Will Denissen, Alok Choudhary, Henry Zongaro, Scott Baden, P. Sadayappan, Steve Hammond, Piyush Mehrota, Barbara Chapman, Joe Fasel.
Group E: "external issues"
enable use of coordination facilities on top of HPF
HPF ext. interface { debugger tables / trace formats}
runtime array management facilities
(e.g. allocate with specified distribution & alignment)
access to data descriptions
HPF Kernel
extrinsic interfaces - local/serial
any multi-language issues
related CCI questions
F95 feedback about interoperabilityDavid Loveman chair, Carol Munroe, Jerry Wagener, Mary Zosel, Andy Meltzer, Jim Cowie.
Each chairperson will designate an co-chair for backup purposes.
The charter of activities for the subgroups was reviewed: make a plan for addressing new features; address current set of CCI issues; assign homework for July meeting.
This concluded the Thursday morning meeting. After lunch the subgroups met all afternoon, with some meetings continuing in the evening.
The Friday morning was chaired by Mary Zosel. Scott Baden first presented a report on a study of the costs of using a uniform fine mesh variation for adaptive mesh refinement applications. He found 40-120% factor for extra memory needed and an overall faction of about a 6-fold increase in the space*time requirements. His paper is available.
It was noted that Fortran 95 still won't have arrays of pointers - primarily because no one has posed it as a real need!
Reports from the three subgroups followed - starting with the Group C report by Rob Schreiber. They will be working on the following set of proposals:
async I/O - Alok and Larry - July
iteration mapping - Chuck and Larry July/Sept.
reductions - Joel and Rob July/Sept.
Parallel, pipelined sections - Jaspal and Ram July / Sept.They processed 5 CCI issues. Two were presented for group discussion.
CCI#3 - forall (i = ) a(i)=f() where f is pure but does an allocate with "stat=" that might fail. If there were no stat= it would be ok, the program would stop, but with stat, it might return a different result. Two proposals were discussed:
a) "stat" in allocate is illegal in a pure procedure
b) allocate is illegal in a pure procedure (since the user can, after all, use a local variable). This was amended to be that ALLOCATE and DEALLOCATE are both illegal in a pure procedure. This proposal passed 20-0-6. Chuck will prepare draft language for the document.
CCI #8 Semantics of a forall are to evaluate all rhs and then store in all lhs, but if the assignment operator is user defined this is under user control, not compiler control. The user's definition might make the order of assignment important. Guy queried how an array assignment was handled in this case. Jerry will take this question to X3J3 about the status with respect to elemental functions. Guy pointed out that for WHERE F90 forbids the defined assignment. Chuck presented a proposal that sounded promising: that the evaluation is as if the rhs were assigned into a temp using the defined assignment operator and then the lhs is a direct copy of the already evaluated values. There was discussion of issues like the type of the temp (same as lhs or rhs?). Action Chuck, Guy, Henry, Jerry to circulate proposed wording for this definition.
Guy Steele made the report on Group D activities.
He first reviewed the CCI issues:
CCI #6 If B is aligned with A, allocatable, and the ALLOCATE is called with both A and B in the same statement, did A happen before B?
- could say both examples wrong
- left to right
- let the compiler sort it out <- this was the preferred answer.
CCI # 12 is a related implicit forward reference. The subcommittee thought that the user should get the statements in the right order.This is somewhat different than the compiler-sort-it-out. The group recommended that we think about this.
CCI #7 - ok as is - anther case of not being able to talk about an inherited template of different rank.
CCI # 9 - a textual change is needed - directives must be in the same module scoping unit.
CCI # 10 - not cci but a request for implementation practice.
CCI # 15 Interesting! No language change but add an examples.
CCI # 8 - no change needed, but will study more closely and reexamine in July.
CCI # 11, 13, 17, 19 need more research.
In discussion the topics for group D, several additional topics were introduced: generalized block distribution, different kinds of pointer, explicit interface rules too hard to understand, shadow overlap areas for arrays for nearest-neighbor codes, processor views, distribution views, CMP_alias facility (constrained form of interprocessor equivalence), processor alignment, and default mappings (maybe on per-rank basis?). Every topic has a name attached.
David Loveman made the Group E report. As homework, the group will establish various liaisons with other groups and tool developers: WG5 (Jerry, David); PCRC (Cowie); PTOOLS as a requirements source, Dan Reeds work and AIMS (Mary). David will provide the CCI write-ups. At the next meeting, we will work on reports from vendors about their interlanguage "callibility". The report will be made to the full HPFF group. The subgroup members will also brush up on the extrinsics definitions and kernel proposal.
For initial proposals, Andy Meltzer will make a first cut for July of interoperability of (HP)F -> C - with a goal of being broad enough to handle X windows. For trace/debug, we will work on a requirements list for July and hope for a September initial proposal.
For kernel HPF, the issue is if we are able to identify HPF features that any implementation should handle efficiently - separated from those that may be inefficient on some reasonable implementations - e.g. those with no interprocedural analysis. The related issues is how these should be documented - should there be a "kernel" hpf, or some other form of documentation. There won't be a proposal before September. In discussion of this later issue, it was noted that it would be a good user's guide - but also noted that it might be useful to the compiler to know that the code was staying within a kernel subset.
For CCI issues addressed by Group E:
CCI #5 - maxloc / minloc - X3J3 already addressed and resolved recently. We will just copy their text.
CCI #14 - resolved: no not two hpf directives.
CCI #20 - this poses a non-trivial problem about how the size of a function result is known and / or declared. Fortran 90 doesn't have any way of saying that the result is "assumed shape/size". After extended discussion, the subgroup decided to propose that a local function can only return a scalar, and that the corresponding extrinsic function return value must be either scalar or an array that is made up from the scalars returned by the locals. In full group discussion, Guy pointed out that it is the DISTRIBUTE directive that is the problem - it lies. There was a lot of discussion, with a basic trend that there should be no change, but rather explanatory text in the document. To bring the issue to close there was a vote on the proposal for no change, but with text explaining the issue - - (choosing whatever reason among several that the individual liked best for no change)/ This passed 14 - 1 - 6.
Break.
After a short break, final details of the meeting were covered. Email will be archived at Mississippi State.
The future meeting location for September and November will be Dallas with the hotel location to be determined.
Attending the May 95 HPFF Meeting:
Robert Babb U. of Denver babb@cs.du.edu Scott Baden UCSD baden@cs.ucsd.edu Ralph Brickner LANL rgb@lanl.gov Barbara Chapman Vienna Univ. barbara@vcpc.univie.a.a? Alok Choudhary Syracuse U. choudhar@cat.syr.edu James Cowie Cooperating Systems cowie@cooperate.com Will Denissen Delft Univ. den_wja@tpp.tno.nl Joe Fasel LANL jhf@lanl.gov Ian Foster Argonne Lab. foster@mcs.anl.gov Steve Hammond NCAR hammond@niwot.scd.ucar.edu Ken Kennedy Rice U./CRPC ken@rice.edu Charles Koelbel Rice U. chk@cs.rice.edu David Loveman Digital loveman@msbcs.enet.dec.com Larry Meadows The Portland Group lfm@pgroup.com Piyush Mehrotra ICASE pm@icase.edu Andy Meltzer Cray Research meltzer@cray.com Carol Munroe Thinking Machines munroe@think.com Carl Offner Digital offner@hpc.pko.dec.com J. Ramanujam Lousiana State University jxr@max.ee.lsu.edu P. Sadayappan Ohio State University saday@cis.ohio-state.edu Joel Saltz U. of Maryland saltz@cs.umd.edu Rob Schreiber RIACS schreiber@riacs.edu Yoshiki Seo NEC yoshiki@cs.rice.edu Guy Steele Sun Microsystems Guy.Steele@East.Sun.Com Jaspal Subhlok Carnegie Mellon jass@cs.emu.edu Paula Vaughan Mississippi St. Univ. paula@erc.msstate.edu Jerry Wagener Amoco jwagener@amoco.com Joel Williamson Convex Computer Corp. joelw@convex.com Henry Zongaro IBM Canada zongaro@vnet.ibm.com Mary Zosel LLNL zosel@llnl.gov