[Openshmem-list] Effects of removing implicit finalize from the specification
Nicholas Park
nspark at computer.org
Tue Mar 21 21:19:52 UTC 2017
I'm very concerned about first-call semantics on shmem_finalize(). I think
this effectively breaks general-purpose library support for OpenSHMEM.
Consider two libraries FOO and BAR that both build on OpenSHMEM.
Claim: Library interoperability has two goals:
1) Libraries FOO and BAR can both initialize and finalize the OpenSHMEM
library
2) Users of FOO need not make OpenSHMEM calls can can interact solely
with FOO
Under the proposed first-call semantic, Goal #1 is violated. A call
sequence of:
FOO_init() --> includes call to shmem_init() --> actually initializes
BAR_init() --> includes call to shmem_init() --> no-op
BAR_finalize() --> includes call to shmem_finalize() --> actually
finalizes
FOO_finalize() --> includes call to shmem_finalize() --> no-op
means that FOO_finalize() will fail if it calls *any* OpenSHMEM function
other than shmem_finalize(). It could non-trivially require calls to
shmem_free(), shmem_my_pe(), shmem_barrier_all().
In contrast, the reference-counted semantic leads to valid behavior:
FOO_init() --> includes call to shmem_init() --> actually initializes
BAR_init() --> includes call to shmem_init() --> no-op
BAR_finalize() --> includes call to shmem_finalize() --> no-op
FOO_finalize() --> includes call to shmem_finalize() --> actually
finalizes
Currently, Goal #2 is violated by the fact that calling shmem_init() more
than once is undefined. Each library must make the assumption that
OpenSHMEM is either initialized/finalized solely by them or external to
them. Only the latter is truly sane, but then means that a user interacting
solely with FOO (without loss of generality) has to make OpenSHMEM calls
(and may not even be aware that FOO relies on OpenSHMEM). I recognize we're
trying to solve that in this discussion, but I wanted to make this goal
explicit for library support.
Nick
On Tue, Mar 21, 2017 at 12:54 PM, Naveen Ravichandrasekaran <nravi at cray.com>
wrote:
> +1 for the first-call semantics – init/finalize on the first-call and then
> no-op after that.
>
>
>
> -Naveen N Ravi.
>
>
>
> *From:* Bob Cernohous
> *Sent:* Tuesday, March 21, 2017 11:09 AM
> *To:* Kuehn, Jeff <jakuehn at lanl.gov>; Naveen Ravichandrasekaran <
> nravi at cray.com>; Jeff Hammond <jeff.science at gmail.com>
> *Cc:* openshmem-list at openshmem.org
> *Subject:* RE: [Openshmem-list] Effects of removing implicit finalize
> from the specification
>
>
>
> I would vote for first-call semantics. The email example wasn’t
> particularly interesting for atexit. The only reason to want
> atexit(shmem_finalize) is that there might be a code path that doesn’t
> explicitly shmem_finalize. I think you want atexit to finalize, not
> decrement a reference count. So init/finalize on first-call and no-op
> after that?
>
>
>
>
>
> *From:* Openshmem-list [mailto:openshmem-list-bounces at openshmem.org
> <openshmem-list-bounces at openshmem.org>] *On Behalf Of *Kuehn, Jeff
> *Sent:* Tuesday, March 21, 2017 10:48 AM
> *To:* Naveen Ravichandrasekaran <nravi at cray.com>; Jeff Hammond <
> jeff.science at gmail.com>
> *Cc:* openshmem-list at openshmem.org
> *Subject:* Re: [Openshmem-list] Effects of removing implicit finalize
> from the specification
>
>
>
> I have a very small concern with nested semantics vs first call semantics
> in the case of init and finalize. If one considers the cases of an extra
> init or finalize and a missing init or finalize, it seems that the
> observable side effects of those bugs, are more local to the problem code,
> and thus easier to debug in the case of first-call semantics, and more
> non-local in the case of nested semantics. It’s a small concern, but what
> do the rest of you think?
>
>
>
> Regards,
>
> Jeff
>
>
>
>
>
>
>
> On 3/20/17, 19:51, "Openshmem-list on behalf of Naveen Ravichandrasekaran"
> <openshmem-list-bounces at openshmem.org on behalf of nravi at cray.com> wrote:
>
>
>
> Hi Jeff,
>
>
>
> > We either need to allow multiple calls to initialize/finalize or we
>
> > need to have is_initialized/is_finalized queries. MPI does the latter
>
> > but since it is utterly trivial to ref count in this case, we should
> just
>
> > do better than MPI and do the former.
>
>
>
> My requirement is not to support multiple shmem_finalize calls. I was just
>
> merely stating the fact that the side-effect of implicit finalize in the
> current
>
> specification allows the usage of shmem_finalize multiple times.
>
>
>
> What we really need is a semantics which successfully performs finalize or
>
> do no-op when the user uses atexit(shmem_finalize) in their application.
>
>
>
> shmem_init(); /* ref_count++ -> actual initialization */
>
> shmem_init(); /* ref_count++ -> no-op */
>
> shmem_finalize(); /* ref_count-- -> no-op */
>
> shmem_finalize(); /* ref_count-- -> actual finalization */
>
>
>
> FWIU, the reference counter method would be useful for handling
>
> multiple shmem_init and shmem_finalize calls, but it won’t probably
>
> work for handling atexit(shmem_finalize).
>
>
>
> In this example, it is unclear how the reference counter works.
>
> int main(void) {
>
> shmem_init(); /* ref_count++ -> actual initialization */
>
> atexit(shmem_finalize);
>
> shmem_finalize(); /* ref_count-- -> actual finalization */
>
> return 0; /* unclear what should happen here
> */
>
> }
>
>
>
> Irrespective of the number of times shmem_init is used, we need to
>
> successfully handle atexit(shmem_finalize). So, I would prefer adding
>
> new APIs in SHMEM to query is_initialized/is_finalized and let the
>
> users handle atexit(shmem_finalize) scenario correctly.
>
>
>
> -Naveen N Ravi.
>
>
>
> *From:* Jeff Hammond [mailto:jeff.science at gmail.com
> <jeff.science at gmail.com>]
> *Sent:* Monday, March 20, 2017 4:01 PM
> *To:* Naveen Ravichandrasekaran <nravi at cray.com>
> *Cc:* openshmem-list at openshmem.org
> *Subject:* Re: [Openshmem-list] Effects of removing implicit finalize
> from the specification
>
>
>
> If we permit multiple calls to finalize then we should also permit
> multiple calls to initialize. This makes it easy on applications that use
> libraries and the libraries do not know whether they should call initialize
> or not.
>
>
>
> We either need to allow multiple calls to initialize/finalize or we need
> to have is_initialized/is_finalized queries. MPI does the latter but since
> it is utterly trivial to ref count in this case, we should just do better
> than MPI and do the former.
>
>
>
> We could add return codes on initialize/finalize to allow users to know
> whether the calls were no-ops or not. This does not break any existing
> code because users do not have to assign return codes to variables.
>
>
>
> Jeff
>
>
>
> On Mon, Mar 20, 2017 at 1:37 PM, Naveen Ravichandrasekaran <nravi at cray.com>
> wrote:
>
> At present, the specification allows the multiple usage of shmem_finalize.
> This seems to be a defined behavior and multiple subsequent calls to
> finalize
> are no-ops.
>
> The following example should pass as per the current specification;
>
> Example:1
> int main(void) {
> shmem_init();
> shmem_finalize();
> shmem_finalize();
> return 0;
> }
>
> In Example:1, we have three finalize (2 explicit + 1 implicit) operations.
> Even any common SHMEM usage, will have two finalize operations
> (1 explicit + 1 implicit). Even though the specification is not clear
> about the
> above usage, this is the side-effect of having implicit finalize at exit()
> or at
> return from main.
>
> Example:2
> int main(void) {
> shmem_init();
> atexit(shmem_finalize);
> return 0;
> }
>
> During F2F meeting, we decided to drop implicit finalize in the
> specification.
> The argument was that the users can explicitly call "atexit" in the
> application
> to achieve the above behavior as shown in Example:2.
>
> Example:3
> int main(void) {
> shmem_init();
> atexit(shmem_finalize);
> shmem_finalize();
> return 0;
> }
>
> FWIU, if we remove implicit finalize from the specification - Example:2 is
> guaranteed to work but the behavior on Example:3 is undefined.
>
> To allow the explicit atexit usage, we either
> 1. Need to specify that multiple calls to finalize are no-ops or
> 2. We need to have a new API - "shmem_finalized()" so that the users can
> have a wrapper from atexit which checks shmem_finalized() before calling
> shmem_finalize().
>
> Please let me know, if this analysis looks valid.
>
> -Naveen N Ravi
> Cray Inc.
>
> _______________________________________________
> Openshmem-list mailing list
> Openshmem-list at openshmem.org
> http://www.openshmem.org/mailman/listinfo/openshmem-list
>
>
>
>
>
> --
>
> Jeff Hammond
> jeff.science at gmail.com
> http://jeffhammond.github.io/
>
> _______________________________________________
> Openshmem-list mailing list
> Openshmem-list at openshmem.org
> http://www.openshmem.org/mailman/listinfo/openshmem-list
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.openshmem.org/pipermail/openshmem-list/attachments/20170321/21f3ea20/attachment-0001.html>
More information about the Openshmem-list
mailing list