Requirements and Preferences
General EIO Requirements
Non-blocking. Normal usage of the EIO
library should not cause a vat to block indefinitely on external I/O,
or block at all on any non-prompt I/O, even when there are bugs in
the E code using EIO.
(It would be pointless to require that malicious usage must not cause
a vat to block, since a malicious entity in a vat can effectively
block its whole hosting vat anyway with an infinite loop.)
When wrapping legacy I/O libraries, this requirement relies on those
libraries stating accurately what operations may proceed promptly.
For example, an InStream
that wraps a Java1.3 InputStream or Reader will
only be prompt to the extent that InputStream.available()
indicates what can be read promptly.
Efficient later. EIO must be implementable
in Java1.4 using NIO
without introducing unnecessary threads.
Possible now. EIO must be implementable
in pure Java1.3 (ie, without NIO
Note that E itself requires
only 1.3 and so must depend on only 1.3. (Though of course, E
must be compatible with 1.4 and later versions.) Note that this requirement
cannot be met efficiently now -- on Java 1.3-based implementation
will require a separate thread to block on each separately blockable
I/O device on which there's a separate I/O operation posted.
InStream and OutStream
The center of any I/O library design are the interfaces often known by
names like InputStream and OutputStream. For EIO, these are InStream
refered to jointly as EIO Streams.
(XXX I find it difficult to write requirements
per se. The following list mixes goals with some description of the actual
design we arrived at in service of those goals.)
As with other I/O libraries, the EIO library both defines the types InStream/OutStream,
for itself and others to implement, and provides some built-in implementations
of these types. When the following list of goals says, for example, that
"A conforming Foo MUST
...", this means, as is conventional, that an implementation
of Foo must have the stated property in order to be considered
conforming, though conformance is not detected or enforced, and so should
not generally be relied upon by clients of a Foo.
Unless stated otherwise, we do require all built in primitive Stream
implementations provided by the EIO library to conform, and the built-in
layered EOI-provided Stream implementations (see Composability
below) to be conformance preserving -- if the layers they're built
on conform, then they must conform as well. Likewise, an EIO-provided
Stream that wraps (is layered upon) a java.io stream must be
conformance preserving -- if the java.io stream conforms to its
contract, then the wrapping EIO Stream must conform as well.
The goals EIO Streams (InStream and OutStream) should
Wraps legacy. EIO Streams must be able
to wrap the common java.io byte and character stream classes,
such that EIO Streams can be used from E
instead of these java.io stream classes with well enough
that the java.io stream classes can be removed from the portable
E API spec without loss
of needed functionality.
Tames legacy. This wrapping must eventually
be sufficient that we can tame
legacy blocking operations out of existence (from the perspective
of the E programmer), or
at least make them unnecessary for normal use.
Objects can play too. Besides bytes
and chars, the API must provide for streams of any type of element,
in order to bring the power of stream-oriented programming to objects.
However, when the type is in fact a scalar
type, then a packed list-of-scalar representation should
be used. When used within a vat, a stream of objects must pass references
to the objects themselves, without serialization.
Simple model. The following is the model
we're using, and serves as an example of the kind of model we require:
A stream is an ongoing sequence of elements of some type, some
of which may be known now, and some of which are expected to become
known in the future. The stream of elements may continue forever,
it may successfully close, or it may fail with a terminal
problem (an IOException explaining the reason for failure).
Whether it closes or fails, it is done with a terminator
indicating how it is done (either true or a reference broken
by the terminal problem). This terminator is conceptually in the
stream after the last element, but it is not itself an element of
Elements enter a stream through an OutStream and exit
through an InStream. When Alice is writing into a stream
that Bob reads, we say that Alice is upstream of Bob. Likewise,
the OutStream is upstream of the InStream. A client
of an OutStream, such as Alice, is a producer. A
client of an InStream, such as Bob, is a consumer.
Synchronous use is instantly familiar.
The synchronous reading operations must be easily understandable to
someone familiar with java.io's InputStream
Likewise with the synchronous writing operations and java.io's OutputStream
Asynchronous use is easily learned.
The asynchronous operations must be easily understandable to someone
who's learned the synchronous operations and is already familiar with
the basic E concurrency
Simple things must be simple, complex
things should be possible. We are explicitly willing to
accept APIs unable to do some complex things, to be examined on a
Fail-stop. The streams that OutStream
writes and InStream reads should be fail-stop. The built-in
streams provided by EIO must be fail-stop. This means they must reliably
deliver the elements of the stream in order until they close or fail.
Failure must be distinguishable from normal termination. As on normal
termination, on failure the stream must stop -- termination must be
a permanent condition. Once a stream is done, it should release its
Pipes. EIO must provide a pipe abstraction
that has an InStream facet and an OutStream
facet. Everything written to the OutStream facet must
be immediately available to from the InStream facet.
Filters. Streams must be easily
composable: As with Unix shell programming, it must be easy to
create filters -- intermediaries that read from an InStream
and write to an OutStream. Placing filters between pipes effectively
makes longer pipes or transducting pipes. Such a filter may or
may not require the InStream and OutStream to be in the same vat
as itself. The EIO library itself must provide a some basic filters,
including an inter-vat copying filter, which can therefore be
used for making inter-vat pipes.
Pipes as "opto-isolators".
As is generally true of capability style, InStreams and OutStreams
should have no unchecked
preconditions. The InStreams and OutStreams provided by EIO
must nave no unchecked preconditions. Therefore, when Alice writes
to the OutStream W of a pipe and Bob reads from the InStream R
of the pipe (Alice->W=R->Bob), then Alice must
see good W behavior even if Bob misbehaves, and Bob must see good
R behavior even if Alice misbehaves.
Failure propogation. Pipes and
filters, including the built-in ones, may spontaneously fail unless
their specific contract says otherwise. They may also be told
to fail by their clients (through the InStream and OutStream interfaces).
The built in pipes and filters must report their failure both
upstream and downstream. A built-in pipe or filter that isn't
already done and receives a failure report must itself fail with
the same terminal problem, whose report it must therefore propogate.
Other pipes and filters should do likewise.
Close propogation. The built-in
pipes and filters must not spontaneously close (terminate
successfully) -- A close indicates an intentional decision
by some client of the EIO library. When Alice closes her OutStream,
she is informing Bob that she has finished writing elements to
the stream. When Bob closes his InStream, he is informing Alice
that he's no longer interested in anything she has to say.
If Carol closes an InStream or OutStream that's in the chain
between Alice and Bob (or likewise, if she tells it to fail),
then she's revoking Alice's ability to talk to Bob through that
channel. The built-in pipes and filters must therefore propogate
closes both upstream and downstream. (For this use, we currently
recommend that Carol tell the stream to fail rather then close.
She can then use the terminal problem to inform both Alice and
Bob why she's revoked their ability to communicate.)
Backpressure, or flow control.
There must be a basic set of built-in pipes and filters that only
require bounded buffers. As these buffers fill up, these must
propogate backpressure upstream, which is to say, avoid draining
elements from the upstream pipes and filters. Therefore, the InStream
and OutStream APIs must make it possible to deal with bounded
buffers and to propogate backpressure. Backpressure must be able
to propogate to the EIO client code at the end point who can use
it to postpone further production.
not. Although comm protocols typically bundle flow control
and error control together, error control (retransmission) is
explicitly not a requirement, since any such errors can be masked
at a lower level, and our fail-stop requirement requires them
to be masked. By contrast, flow control cannot be masked, but
must instead be made visible to application code.
Flush pressure. With an OutStream,
it must be possible to obligate a stream to eventually deliver
all elements that have already entered the stream. The built-in
pipes and filters must honor this obligation up to the limits
set by termination and backpressure.
Preserve immediacy. The built-in legacy
wrappers must provide all the immediate reading or writing power of
the underlying stream abstractions: If 37 bytes are available on the
underlying stream, then it must be possible to read 37 bytes now
from an InStream that wraps that stream. When composed through pipes
and filters, this immediacy may be subject to the limitations imposed
by these intermediaries. In particular, all immediacy guarantees are
lost over an inter-vat pipe.