[manpage_begin comm n 4.2]
[copyright {1995-1998 The Open Group. All Rights Reserved.}]
[copyright {2003 ActiveState Corporation.}]
[moddesc   {remote communication}]
[titledesc {A remote communications facility for Tcl (7.6, 8.0, and later)}]
[require Tcl 8.2]
[require comm [opt 4.2]]
[description]

[para]

The [package comm] command provides an inter-interpreter remote
execution facility much like Tk's [cmd send(n)], except that it uses
sockets rather than the X server for the communication path.  As a
result, [package comm] works with multiple interpreters, works on
Windows and Macintosh systems, and provides control over the remote
execution path.

[para]

These commands work just like [cmd send] and [cmd "winfo interps"] :

[example {
 ::comm::comm send ?-async? id cmd ?arg arg ...?
 ::comm::comm interps
}]

[para]

This is all that is really needed to know in order to use
[package comm]

[section  COMMANDS]
[para]

The package initializes [cmd ::comm::comm] as the default [emph chan].

[para]
[package comm] names communication endpoints with an [emph id] unique
to each machine.  Before sending commands, the [emph id] of another
interpreter is needed.  Unlike Tk's send, [package comm] doesn't
implicitly know the [emph id]'s of all the interpreters on the system.

The following four methods make up the basic [package comm] interface.

[list_begin definitions]

[call [cmd "::comm::comm send"] [opt -async] [opt "-command [arg callback]"] [arg id] [arg cmd] [opt "[arg "arg arg ..."]"]]

This invokes the given command in the interpreter named by [arg id].  The
command waits for the result and remote errors are returned unless the
[option -async] or [option -command] option is given.  If [option -async]
is given, send returns immediately and there is no further notification of
result.  If [option -command] is used, [emph callback] specifies a command
to invoke when the result is received.  These options are mutually
exclusive.  The callback will receive arguments in the form
[emph "-option value"], suitable for [cmd "array set"].
The options are: [emph -id], the comm id of the interpreter that received
the command; [emph -serial], a unique serial for each command sent to a
particular comm interpreter; [emph -chan], the comm channel name;
[emph -code], the result code of the command; [emph -errorcode], the
errorcode, if any, of the command; [emph -errorinfo], the errorinfo, if
any, of the command; and [emph -result], the return value of the command.
If connection is lost before a reply is received, the callback will be
invoked with a connection lost message with -code equal to -1.  When
[option -command] is used, the command returns the unique serial for the
command.

[call [cmd "::comm::comm self"]]

Returns the [emph id] for this channel.

[call [cmd "::comm::comm interps"]]

Returns a list of all the remote [emph id]'s to which this channel is
connected.  [package comm] learns a new remote [emph id] when a command
is first issued it, or when a remote [emph id] first issues a command
to this comm channel.  [cmd "::comm::comm ids"] is an alias for this
method.

[call [cmd "::comm::comm connect"] [opt [arg id]]]

Whereas [cmd "::comm::comm send"] will automatically connect to the
given [arg id], this forces a connection to a remote [emph id] without
sending a command.  After this, the remote [emph id] will appear in
[cmd "::comm::comm interps"].

[list_end]

[section  "EVAL SEMANTICS"]
[para]

The evaluation semantics of [cmd "::comm::comm send"] are intended to
match Tk's [cmd send] [emph exactly]. This means that [package comm]
evaluates arguments on the remote side.

[para]

If you find that [cmd "::comm::comm send"] doesn't work for a
particular command, try the same thing with Tk's send and see if the
result is different.  If there is a problem, please report it.  For
instance, there was had one report that this command produced an
error.  Note that the equivalent [cmd send] command also produces the
same error.

[para]
[example {
 % ::comm::comm send id llength {a b c}
 wrong # args: should be "llength list"
 % send name llength {a b c}
 wrong # args: should be "llength list"
}]

[para]

The [cmd eval] hook (described below) can be used to change from
[cmd send]'s double eval semantics to single eval semantics.

[section  "MULTIPLE CHANNELS"]
[para]

More than one [cmd comm] channel (or [emph listener]) can be created
in each Tcl interpreter.  This allows flexibility to create full and
restricted channels.  For instance, [term hook] scripts are specific
to the channel they are defined against.

[list_begin definitions]

[call [cmd "::comm::comm new"] [arg chan] [opt [arg "name value ..."]]]

This creates a new channel and Tcl command with the given channel
name.  This new command controls the new channel and takes all the
same arguments as [cmd ::comm::comm].  Any remaining arguments are
passed to the [cmd config] method.  The fully qualified channel
name is returned.

[call [cmd "::comm::comm channels"]]

This lists all the channels allocated in this Tcl interpreter.

[list_end]
[para]

The default configuration parameters for a new channel are:

[para]
[example {
  "-port 0 -local 1 -listen 0"
}]

[para]

The default channel [cmd ::comm::comm] is created with:

[para]
[example {
  "::comm::comm new ::comm::comm -port 0 -local 1 -listen 1"
}]

[section  "CHANNEL CONFIGURATION"]
[para]

The [cmd config] method acts similar to [cmd fconfigure] in that it
sets or queries configuration variables associated with a channel.

[list_begin definitions]
[call [cmd "::comm::comm config"]]
[call [cmd "::comm::comm config"] [arg name]]
[call [cmd "::comm::comm config"] [opt [arg "name value ..."]]]

When given no arguments, [cmd config] returns a list of all variables
and their value With one argument, [cmd config] returns the value of
just that argument.  With an even number of arguments, the given
variables are set to the given values.

[list_end]

[para]

These configuration variables can be changed (descriptions of them are
elsewhere in this manual page):

[list_begin definitions]
[lst_item "[option -listen] [opt [arg 0|1]]"]
[lst_item "[option -local]  [opt [arg 0|1]]"]
[lst_item "[option -port]   [opt [arg port]]"]
[list_end]

[para]
These configuration variables are readonly:

[list_begin definitions]
[lst_item "[option -chan]    [arg chan]"]
[lst_item "[option -serial]  [arg n]"]
[lst_item "[option -socket]  sock[arg In]"]
[list_end]

[para]

When [cmd config] changes the parameters of an existing channel, it
closes and reopens the listening socket.  An automatically assigned
channel [emph id] will change when this happens.  Recycling the socket
is done by invoking [cmd "::comm::comm abort"], which causes all
active sends to terminate.

[section  "ID/PORT ASSIGNMENTS"]
[para]

[package comm] uses a TCP port for endpoint [emph id].  The

[method interps] (or [method ids]) method merely lists all the TCP ports
to which the channel is connected.  By default, each channel's

[emph id] is randomly assigned by the operating system (but usually
starts at a low value around 1024 and increases each time a new socket
is opened).  This behavior is accomplished by giving the

[option -port] config option a value of 0.  Alternately, a specific
TCP port number may be provided for a given channel.  As a special
case, comm contains code to allocate a a high-numbered TCP port
(>10000) by using [option "-port {}"].  Note that a channel won't be
created and initialized unless the specific port can be allocated.

[para]

As a special case, if the channel is configured with

[option "-listen 0"], then it will not create a listening socket and
will use an id of [emph 0] for itself.  Such a channel is only good
for outgoing connections (although once a connection is established,
it can carry send traffic in both directions).

[section  "REMOTE INTERPRETERS"]
[para]

By default, each channel is restricted to accepting connections from
the local system.  This can be overridden by using the

[option "-local 0"] configuration option For such channels, the

[emph id] parameter takes the form [emph "\{ id host \}"].

[para]

[emph WARNING]: The [emph host] must always be specified in the same
form (e.g., as either a fully qualified domain name, plain hostname or
an IP address).

[section  "CLOSING CONNECTIONS"]
[para]

These methods give control over closing connections:

[list_begin definitions]

[call [cmd "::comm::comm shutdown"] [arg Iid]]

This closes the connection to [arg id], aborting all outstanding
commands in progress.  Note that nothing prevents the connection from
being immediately reopened by another incoming or outgoing command.

[call [cmd "::comm::comm abort"]]

This invokes shutdown on all open connections in this comm channel.

[call [cmd "::comm::comm destroy"]]

This aborts all connections and then destroys the this comm channel
itself, including closing the listening socket.  Special code allows
the default [cmd ::comm::comm] channel to be closed such that the

[cmd ::comm::comm] command it is not destroyed.  Doing so closes the
listening socket, preventing both incoming and outgoing commands on
the channel.  This sequence reinitializes the default channel:

[nl]
[example {
 "::comm::comm destroy; ::comm::comm new ::comm::comm"
}]

[list_end]

[para]

When a remote connection is lost (because the remote exited or called
[cmd shutdown]), [package comm] can invoke an application callback.
This can be used to cleanup or restart an ancillary process, for
instance.  See the [term lost] callback below.

[section  CALLBACKS]
[para]
This is a mechanism for setting hooks for particular events:

[list_begin definitions]

[call [cmd "::comm::comm hook"] [arg event] [opt [const +]] [opt [arg script]]]

This uses a syntax similar to Tk's [cmd bind] command.  Prefixing

[arg script] with a [const +] causes the new script to be appended.
Without this, a new [arg script] replaces any existing script.  When
invoked without a script, no change is made.  In all cases, the new
hook script is returned by the command.

[nl]

When an [arg event] occurs, the [arg script] associated with it is
evaluated with the listed variables in scope and available.  The
return code ([emph not] the return value) of the script is commonly
used decide how to further process after the hook.

[nl]
Common variables include:

[list_begin definitions]

[lst_item [var chan]]
the name of the comm channel (and command)

[lst_item [var id]]
the id of the remote in question

[lst_item [var fid]]
the file id for the socket of the connection

[list_end]
[list_end]

[para]
These are the defined [emph events]:

[list_begin definitions]

[lst_item [const connecting]]

Variables:
[arg "chan id host port"]

[nl]

This hook is invoked before making a connection to the remote named in
[arg id].  An error return (via [cmd error]) will abort the connection
attempt with the error.  Example:

[nl]
[example {
 % ::comm::comm hook connecting {
     if [lb]string match {*[lb]02468[rb]} $id[rb] {
         error "Can't connect to even ids" 
     }
 }
 % ::comm::comm send 10000 puts ok
 Connect to remote failed: Can't connect to even ids
 % 
}]

[lst_item [const connected]]

Variables:
[arg "chan fid id host port"]
[nl]

This hook is invoked immediately after making a remote connection to
[arg id], allowing arbitrary authentication over the socket named by
[arg fid].  An error return (via [cmd error] ) will close the
connection with the error.  [arg host] and [arg port] are merely
extracted from the [arg id]; changing any of these will have no effect
on the connection, however.  It is also possible to substitute and
replace [arg fid].

[lst_item [const incoming]]

Variables:
[arg "chan fid addr remport"]
[nl]

Hook invoked when receiving an incoming connection, allowing arbitrary
authentication over socket named by [arg fid].  An error return (via
[cmd error]) will close the connection with the error.  Note that the
peer is named by [arg remport] and [arg addr] but that the remote
[emph id] is still unknown.  Example:

[nl]
[example {
 ::comm::comm hook incoming {
     if [lb]string match 127.0.0.1 $addr[rb] {
         error "I don't talk to myself"
     }
 }
}]


[lst_item [const eval]]

Variables:
[arg "chan id cmd buffer"]
[nl]

This hook is invoked after collecting a complete script from a remote
but [emph before] evaluating it.  This allows complete control over
the processing of incoming commands.  [arg cmd] contains either
[const send] or [const async].  [arg buffer] holds the script to
evaluate.  At the time the hook is called, [arg "\$chan remoteid"] is
identical in value to [arg id].

[nl]

By changing [arg buffer], the hook can change the script to be
evaluated.  The hook can short circuit evaluation and cause a value to
be immediately returned by using [cmd return] [arg result] (or, from
within a procedure, [cmd "return -code return"] [arg result]).  An
error return (via [cmd error]) will return an error result, as is if
the script caused the error.  Any other return will evaluate the
script in [arg buffer] as normal.  For compatibility with 3.2,

[cmd break] and [cmd "return -code break"] [arg result] is supported,
acting similarly to [cmd "return {}"] and [cmd "return -code return"]
[arg result].

[nl]

Examples:

[list_begin enum]

[enum]
augmenting a command
[nl]
[example {
 % ::comm::comm send [lb]::comm::comm self[rb] pid
 5013
 % ::comm::comm hook eval {puts "going to execute $buffer"}
 % ::comm::comm send [lb]::comm::comm self[rb] pid
 going to execute pid
 5013
}]

[enum]
short circuiting a command
[nl]
[example {
 % ::comm::comm hook eval {puts "would have executed $buffer"; return 0}
 % ::comm::comm send [lb]::comm::comm self[rb] pid
 would have executed pid
 0
}]

[enum]
Replacing double eval semantics
[nl]
[example {
 % ::comm::comm send [lb]::comm::comm self[rb] llength {a b c}
 wrong # args: should be "llength list"
 % ::comm::comm hook eval {return [uplevel #0 $buffer]}
 return [lb]uplevel #0 $buffer[rb]
 % ::comm::comm send [lb]::comm::comm self[rb] llength {a b c}
 3
}]

[enum]
Using a slave interpreter
[nl]
[example {
 % interp create foo
 % ::comm::comm hook eval {return [lb]foo eval $buffer[rb]}
 % ::comm::comm send [lb]::comm::comm self[rb] set myvar 123
 123
 % set myvar
 can't read "myvar": no such variable
 % foo eval set myvar
 123
}]

[enum]
Using a slave interpreter (double eval)
[nl]
[example {
 % ::comm::comm hook eval {return [lb]eval foo eval $buffer[rb]}
}]

[enum]
Subverting the script to execute
[nl]
[example {
 % ::comm::comm hook eval {
     switch -- $buffer {
         a {return A-OK} b {return B-OK} default {error "$buffer is a no-no"}
     }
 }
 % ::comm::comm send [lb]::comm::comm self[rb] pid
 pid is a no-no
 % ::comm::comm send [lb]::comm::comm self[rb] a
 A-OK
}]

[list_end]


[lst_item [const reply]]

Variables:
[arg "chan id buffer ret return()"]
[nl]

This hook is invoked after collecting a complete reply script from a
remote but [emph before] evaluating it.  This allows complete
control over the processing of replies to sent commands.  The reply
[arg buffer] is in one of the following forms

[list_begin bullet]
[bullet]
return result
[bullet]
return -code code result
[bullet]
return -code code -errorinfo info -errorcode ecode msg
[list_end]

For safety reasons, this is decomposed.  The return result is in
[arg ret], and the return switches are in the return array:

[list_begin bullet]
[bullet]
[emph return(-code)]
[bullet]
[emph return(-errorinfo)]
[bullet]
[emph return(-errordcode)]
[list_end]

Any of these may be the empty string.  Modifying these four variables
can change the return value, whereas modifying [arg buffer] has no
effect.

[lst_item [const callback]]

Variables:
[arg "chan id buffer ret return()"]
[nl]

Similar to [emph reply], but used for callbacks.

[lst_item [const lost]]

Variables:
[arg "chan id reason"]
[nl]

This hook is invoked when the connection to [emph id] is lost.  Return
value (or thrown error) is ignored.  [arg reason] is an explanatory
string indicating why the connection was lost.  Example:

[nl]

[example {
 ::comm::comm hook lost {
     global myvar
     if {$myvar(id) == $id} {
         myfunc
         return
     }
 }
}]

[list_end]

[section  UNSUPPORTED]
[para]
These interfaces may change or go away in subsequence releases.

[list_begin definitions]
[call [cmd "::comm::comm remoteid"]]

Returns the [emph id] of the sender of the last remote command
executed on this channel.  If used by a proc being invoked remotely,
it must be called before any events are processed.  Otherwise, another
command may get invoked and change the value.

[call [cmd ::comm::comm_send]]

Invoking this procedure will substitute the Tk [cmd send] and
[cmd "winfo interps"] commands with these equivalents that use
[cmd ::comm::comm].

[nl]

[example {
 proc send {args} {
     eval ::comm::comm send $args
 }
 rename winfo tk_winfo
 proc winfo {cmd args} {
     if ![lb]string match in* $cmd[rb] {return [lb]eval [lb]list tk_winfo $cmd[rb] $args[rb]}
     return [lb]::comm::comm interps[rb]
 }
}]

[list_end]

[section  SECURITY]
[para]
Something here soon.

[section "BLOCKING SEMANTICS"]

[para]

There is one outstanding difference between [package comm] and

[cmd send].  When blocking in a synchronous remote command, [cmd send]
uses an internal C hook (Tk_RestrictEvents) to the event loop to look
ahead for send-related events and only process those without
processing any other events.  In contrast, [package comm] uses the

[cmd vwait] command as a semaphore to indicate the return message has
arrived.  The difference is that a synchronous [cmd send] will block
the application and prevent all events (including window related ones)
from being processed, while a synchronous [cmd {::comm::comm send}] will block the
application but still allow other events will still get processed.  In
particular, [cmd "after idle"] handlers will fire immediately when
comm blocks.

[para]

What can be done about this?  First, note that this behavior will come
from any code using [cmd vwait] to block and wait for an event to
occur.  At the cost of multiple channel support, [package comm] could
be changed to do blocking I/O on the socket, giving send-like blocking
semantics.  However, multiple channel support is a very useful feature
of comm that it is deemed too important to lose.  The remaining
approaches involve a new loadable module written in C (which is
somewhat against the philosophy of [cmd comm ]) One way would be to
create a modified version of the [cmd vwait] command that allow the
event flags passed to Tcl_DoOneEvent to be specified.  For [cmd comm],
just the TCL_FILE_EVENTS would be processed.  Another way would be to
implement a mechanism like Tk_RestrictEvents, but apply it to the Tcl
event loop (since [package comm] doesn't require Tk).  One of these
approaches will be available in a future [package comm] release as an
optional component.

[section COMPATIBILITY]
[para]

[package comm] exports itself as a package.  The package version number
is in the form [emph "major . minor"], where the major version will
only change when a non-compatible change happens to the API or
protocol.  Minor bug fixes and changes will only affect the minor
version.  To load [package comm] this command is usually used:

[para]
[example {
 package require comm 3
}]

[para]
Note that requiring no version (or a specific version) can also be done.

[para]
The revision history of [package comm] includes these releases:

[list_begin definitions]

[lst_item "4.2"]

Bugfixes, and most important, switched to utf-8 as default encoding
for full i18n without any problems.

[lst_item "4.1"]

Rewrite of internal code to remove old pseudo-object model.  Addition
of send -command asynchronous callback option.

[lst_item "4.0"]

Per request by John LoVerso. Improved handling of error for async
invoked commands.

[lst_item "3.7"]

Moved into tcllib and placed in a proper namespace.

[lst_item "3.6"]

A bug in the looking up of the remoteid for a executed command could
be triggered when the connection was closed while several asynchronous
sends were queued to be executed.

[lst_item "3.5"]

Internal change to how reply messages from a [cmd send] are handled.
Reply messages are now decoded into the [arg value] to pass to

[cmd return]; a new return statement is then cons'd up to with this
value.  Previously, the return code was passed in from the remote as a
command to evaluate.  Since the wire protocol has not changed, this is
still the case.  Instead, the reply handling code decodes the

[const reply] message.

[lst_item "3.4"]

Added more source commentary, as well as documenting config variables
in this man page.  Fixed bug were loss of connection would give error
about a variable named [var pending] rather than the message about
the lost connection.  [cmd "comm ids"] is now an alias for

[cmd "comm interps"] (previously, it an alias for [cmd "comm chans"]).
Since the method invocation change of 3.0, break and other exceptional
conditions were not being returned correctly from [cmd "comm send"].
This has been fixed by removing the extra level of indirection into
the internal procedure [cmd commSend].  Also added propagation of
the [arg errorCode] variable.  This means that these commands return
exactly as they would with [cmd send]:

[example {
 comm send id break
 catch {comm send id break}
 comm send id expr 1 / 0
}]

[nl]

Added a new hook for reply messages.  Reworked method invocation to
avoid the use of comm:* procedures; this also cut the invocation time
down by 40%.  Documented [cmd "comm config"] (as this manual page
still listed the defunct [cmd "comm init"]!)


[lst_item "3.3"]

Some minor bugs were corrected and the documentation was cleaned up.
Added some examples for hooks.  The return semantics of the [cmd eval]
hook were changed.

[lst_item "3.2"]

A new wire protocol, version 3, was added.  This is backwards
compatible with version 2 but adds an exchange of supported protocol
versions to allow protocol negotiation in the future.  Several bugs
with the hook implementation were fixed.  A new section of the man
page on blocking semantics was added.

[lst_item "3.1"]

All the documented hooks were implemented.  [cmd commLostHook] was
removed.  A bug in [cmd "comm new"] was fixed.

[lst_item "3.0"]

This is a new version of [package comm] with several major changes.
There is a new way of creating the methods available under the

[cmd comm] command.  The [cmd "comm init"] method has been retired
and is replaced by [cmd "comm configure"] which allows access to many
of the well-defined internal variables.  This also generalizes the
options available to [cmd "comm new"].  Finally, there is now a
protocol version exchanged when a connection is established.  This
will allow for future on-wire protocol changes.  Currently, the
protocol version is set to 2.

[lst_item "2.3"]

[cmd "comm ids"] was renamed to [cmd "comm channels"].  General
support for [cmd "comm hook"] was fully implemented, but only the
[term lost] hook exists, and it was changed to follow the general
hook API.  [cmd commLostHook] was unsupported (replaced by

[cmd "comm hook lost"]) and [cmd commLost] was removed.

[lst_item "2.2"]

The [term died] hook was renamed [term lost], to be accessed by
[cmd commLostHook] and an early implementation of
[cmd "comm lost hook" ].  As such, [cmd commDied] is now
[cmd commLost].

[lst_item "2.1"]
Unsupported method [cmd "comm remoteid"] was added.

[lst_item "2.0"]
[package comm] has been rewritten from scratch (but is fully compatible
with Comm 1.0, without the requirement to use obTcl).

[list_end]

[section  AUTHOR]

John LoVerso, John@LoVerso.Southborough.MA.US

[para]

[emph http://www.opengroup.org/~loverso/tcl-tk/#comm]


[section  LICENSE]

Please see the file [emph comm.LICENSE] that accompanied this source,
or
[uri http://www.opengroup.org/www/dist_client/caubweb/COPYRIGHT.free.html].

[para]

This license for [package comm], new as of version 3.2, allows it to be
used for free, without any licensing fee or royalty.


[section BUGS]
[list_begin bullet]
[bullet]

If there is a failure initializing a channel created with
[cmd "::comm::comm new"], then the channel should be destroyed.
Currently, it is left in an inconsistent state.

[bullet]

There should be a way to force a channel to quiesce when changing the
configuration.

[list_end]

[para]
The following items can be implemented with the existing hooks and are
listed here as a reminder to provide a sample hook in a future
version.

[list_begin bullet]
[bullet]

Allow easier use of a slave interp for actual command execution
(especially when operating in "not local" mode).

[bullet]

Add host list (xhost-like) or "magic cookie" (xauth-like)
authentication to initial handshake.

[list_end]

[para]
The following are outstanding todo items.

[list_begin bullet]
[bullet]

Add an interp discovery and name->port mapping.  This is likely to be
in a separate, optional nameserver.  (See also the related work,
below.)

[bullet]

Fix the [emph "{id host}"] form so as not to be dependent upon
canonical hostnames.  This requires fixes to Tcl to resolve hostnames!

[list_end]

[para]
This man page is bigger than the source file.


[section  "ON USING OLD VERSIONS OF TCL"]

[para]
Tcl7.5 under Windows contains a bug that causes the interpreter to
hang when EOF is reached on non-blocking sockets.  This can be
triggered with a command such as this:

[para]
[example {
 "comm send $other exit"
}]

[para]
Always make sure the channel is quiescent before closing/exiting or
use at least Tcl7.6 under Windows.

[para]
Tcl7.6 on the Mac contains several bugs.  It is recommended you use
at least Tcl7.6p2.

[para]
Tcl8.0 on UNIX contains a socket bug that can crash Tcl.  It is recommended
you use Tcl8.0p1 (or Tcl7.6p2).


[section "RELATED WORK"]
[para]
Tcl-DP provides an RPC-based remote execution interface, but is a
compiled Tcl extension.  See
[emph http://www.cs.cornell.edu/Info/Projects/zeno/Projects/Tcl-DP.html].

[para]
Michael Doyle <miked@eolas.com> has code that implements the Tcl-DP
RPC interface using standard Tcl sockets, much like [package comm]

[para]
Andreas Kupries <andreas_kupries@users.sourceforge.net> uses
[package comm] and has built a simple nameserver as part of his Pool
library.  See [uri http://www.purl.org/net/akupries/soft/pool/index.htm].

[see_also send(n)]
[manpage_end]

