PalCom Language Interop Redesign 2006-06-29

* Description

Component interface is specified using IDA declarations.  IDA
declarations specify class interfaces, a class interface is a
collection of operations.  IDA declarations can be written in separate
files and included into the pcs file or can be written inlined into
the pcs file.  An IDA declaration is always included as metainfo in
the .prc file of a component if the component implements this
interface.  (Note: it will probably be useful to associate IDA
declarations with some kind of namespace - probably an interface can
belong to a given component, but this remains to be resolved.)

IDA interfaces do not support overloading.  The basic method name must
be unique.  Languages that support overloading must use method
renaming to make overloaded methods visible to other languages.
Features that require overloading, such as constructors in Java, can
be handled by introducing a special annotation in IDA such as
"constructor" to indicate that all methods tagget with this annotation
are considered to be constructors.  (Note that a clash between two
different methods for the first part of the Smalltalk selector is
considered to be overloading.)

To make a class visible to other languages, this class must implement
an IDA interface.  Each method in the IDA interface must be present in
the implementation.  Matching the IDA method to the implementation
allows the full signature to be associated with the IDA method.  Since
there is no overloading in IDA and overloading in the source language
is required to be resolved, there is no ambiguity.  

In terms of implementation, methods still retain their
language-specific mangled name, but all such mangled names are
prefixed by a language-specific prefix.  (Currently 'S' for Smalltalk,
'J' for Java, and 'B' for Beta - this prefix can easily be generalized
to arbitrary length prefixes for naming and debugging, so long as the
byte sequence of prefixes is unique.)  The language-specific mangled
names are referred to as the internal name, as opposed to the IDA
signature which is language-independent.

The invoke bytecode uses the internal name.  Dispatching first checks
if this name is in the cache, if yes then the dispatch is performed.
If no, then the internal names of methods in the local class are
searched for a match.  If there is a match, then the call originated
from within the same language, and can be dispatched.  This allows
methods to have a different name in their own language (the internal
name) than the name in the signature.  If there is no match, then the
signature of each method in the local class is mangled and then
compared to the internal name.  If there is a match, the target has
been found, and can be dispatched.  If there is no match, searching
continues in the superclass.  If no match was found,
doesNotUnderstand: is called.  If a match was found and dispatching
continued with a method, this method is added to the cache.  Note that
the mangling is language-specific; the choice of mangler algorithm can
be deduced from the method prefix (e.g, the internal name which was
argument to the invoke bytecode).  The mangling algorithm will as a
first step be implemented internally in the VM.

* Future extensions:

- Component interface: the ability to define an interface for an
  entire component (e.g., "Storage") and then indicate in the PCS file
  that we implement this interface.

- Language-neutral mangling: either let the dispatch mangling be
  written in Smalltalk (perhaps as a separate coroutine which must not
  be interrupted?) or use a small language to define the mangling
  (almost like a printf formatting string).

- Factories: classes are referred to by their implementation names
  when creating instances.  Nevertheless, interface constructors could
  perhaps act as factories, or components could act as factories, in
  both cases meaning "give me some class implementing interface I".

* Example:

IDA interface:

  interface List {
    @constructor init(Integer size);
    @constructor initEmpty();
    Integer size();
    void append(List other);
    Object get(Integer);
    void set(Integer,Object);
  }

(The interface does not include the parameter names since they are not
needed, but does include the return type to allow separate compilation
of typed languages that use classes known only by their interface.)

Smalltalk implementation:

  "@implements: List@"
  SmalltalkList = (
    ...

    size = ( ... )

    append: other = ( ... )

    "@name: get@"
    at: position = ( ... )

    "@name: set@"
    at: position put: object = ( ... )

  )

The Java implementation (for example, using an annotation to indicate
IDA interface implementation - we could probably just use the standard
Java "implements" keyword if we (Torbjorn ;-) are clever):

  @IDAimplements("List") class JavaList {
    List(int size),
    List();
    int size() { ... }
    @IDAname("append") void append(List other) { ... }
    void append(Object element) { ... }  // note: not visible to others
    Object get(int index) { ... }
    void set(int index,Object obj) {... }
  }

The Beta implementation is not included here, but is not problematic
since there is no overloading.

The above examples showed implementing an interface.  Using an
interface should work in the same way for Java.  From Smalltalk
selectors do not have names, which makes a difference when there is
more than one parameter.  For example, the method 'set' from above is
accessible from Smalltalk using the selector 'set::', which can be
used like so: 'set: 7 :someObject'.  (If we want nice selector names
for the remaining parameters then we could include parameter names in
the IDA signatures and use that for matching.  This is however an
implementation detail, and we haven't decided which approach to use.
What to do should probably be motivated by concrete examples.)

* Before and after (language insulation code):

Note: "around" method combination appears to subsume before/after, but
this is actually not what we want for method insulation.  To check
method parameter types, we want to run code specified by the callee.
Conversely, to check method return types, we want to run code specific
by the caller.  Thus, we want the "before" code to be specified by the
callee, and the "after" code to be specified by the caller.  To allow
code to be specified by the caller appears to require the ability to
introduce new methods into the class of the callee.

Before and after can in general be implemented as follows using the
signature dispatch mechanism: instead of mapping a matching method to
the signature implementation, the method is mapped to a wrapper which
calls the target method after performing pre-method processing, or
which can perform post-processing on the target method.  In essence,
since the name mapper controls how the dispatch takes place, it can
redirect the dispatch as appropriate.  Getting the right interaction
between before and after methods will probably be difficult, but the
mechanism should be general enough to support it.

* Inheritance

Inheritance is resolved by the basic mechanism outlined above - if a
method is overridden then the local cache entries will point to the
local method, if a method has not been overridden then the local cache
entries will point to the method from the superclass.  (Similarly,
when doing a self send to an overridden method, the local method cache
will point to the method implemented in the subclass.)  Thus, so long
as the signatures always match when overriding, inheritance will work.
Cross-language inheritance is thus driven by the IDA interfaces, which
seems perfectly appropriate.  (Note: care must be taken to ensure that
false matches to the language-internal name from an overridden method
does not cause an overriding method to be ignored.)

* Comparison with FLIF-1,2

FLIF-1 and FLIF-2 were both based on method aliases, that is,
resolving all language interoperability at load-time, and embedding
the resolution into method naming using aliases.  This can for example
create problems with inheritance, where the whole method group needs
to be overridden.  This approach (FLIF-3?) can have an arbitrarily
complex means of deciding what method to call for a given method name,
since it does not need to be embedded into the method names.
Moreover, this approach can be used on-demand as the program is
running, which avoids wasting space on unused aliases (which can
potentially be quadratic: number of methods times number of
languages).

* A note on mangling and manual override of method names

It is essential that the programmer can manually override the default
mangling.  Overriding means that the generated signature is different
from what the language uses internally.  Always storing the
language-internal name along with the full signature in the class
allows dispatching within the language to still work regardless of the
new signature.

In practice, looking at the Smalltalk and Java libraries currently
implemented for PalCom, there are very few places where overriding
method names is needed.
