#define S_DOCUMENTATION
#define S_DOCUMENTATION_DEVELOPER
#define PAGE Documentation
#define SUBPAGE Developer
#define TITLE Application Data Transmission

#include "header.shtml"

<p>
  Author: <a href="mailto:eilemann@gmail.com">eilemann@gmail.com</a><br/>
  State: Design
</p>

<h2>Overview</h2>
<p>
  The data transmission API is a simple interface to send data from the
  application to the render clients. It sends unstructured data directly to
  running nodes. The main purpose is to efficiently transmit the model database
  during initialization.
</p>
<p>
  The implementation has been deferred, since most of its use can be implemented
  using distributed objects.
</p>

<h2>Design</h2>
<p>
  The API has to provide an efficient way to send data. The data is not
  frame-synchronized, that is, latency is not accounted for. For frame-specific
  data the usage of versioned <code>eq::net::Object</code>s is highly
  recommended. Synchronization of data to config frames can also be implemented
  by the application, using the current frame number.
</p>
<p>
  One approach is not to provide a new API, but to use the command packet
  API. This requires getting a list of nodes from the 'client' API in order to
  know where to send it to. The proposed API is easier to use and can implement
  optimizations later on.
</p>
<p>
  The local (app) node also receives the broadcasted data. This allows for
  the same code to be deployed on the app node. Furthermore, when initializing
  the model, the application node might want to initialize a copy of the model
  as well. If not, it can always discard the received data.
</p>
<p>
  The received data is enqueued by the receiver thread in a separate
  CommandQueue. The CommandQueue transfers the received packet, thus avoiding a
  data copy. To access the data from the enqueued Commands, a wrapper API around
  the CommandQueue is provided. Handling in the receiver thread without queueing
  can be implemented by overwriting the command handler (advanced usage).
</p>
<p>
  Potential overlap with the to-be-defined frame data API (a.k.a. cull queues)
  exists. The frame data API will most likely require objects to provide
  additional information for frustum and range culling. It also requires frame
  synchronization and therefore queueing.
</p>

<h2>Usability during Config::init</h2>
<p>
  As is, the data transmission API can not be used during
  <code>Config::init</code>, because the application is blocked until all
  initialization task methods have returned.
</p>
<p>
  In order to allow data transmission during initialization,
  <code>Config::init</code> is split into <code>startInit</code> and
  <code>finishInit</code>, similar to the config's frame functions. The start
  method returns as soon as all the nodes have been constructed. The finish
  function blocks until all initialization functions have returned, and returns
  the success value of the config initialization. <code>Config::init</code>
  becomes a convenience function calling the start and finish method.
</p>
<p>
  The pseudo-code for sending data during initialization is:
</p>
<pre>
[Application:]
    config->startInit();
    for each init data
        config->broadcastData( data, size );
    config->finishInit();

[Render Clients]
    bool Node::configInit(...)
    {
        ...
        while not all init data received
            data = receiveData(...);
            process data
        ...
    }
</pre>

<h2>Implementation</h2>
<ul>
  <li>Sender Implementation
    <ul>
      <li>config init reply returns list of node IDs</li>
      <li>broadcast connects all nodes and caches list of nodes</li>
      <li>broadcast sends ConfigDataPacket to all connected nodes</li>
    </ul>
  </li>
  <li>Receiver Implementation
    <ul>
      <li>subclass CommandQueue into DataQueue</li>
      <li>push Commands into DataQueue from receiver thread</li>
      <li>return pointer and size of current Command from DataQueue</li>
    </ul>
  </li>
</ul>

<h2>API</h2>
<!-- /* --><pre>
  void eq::Config::broadcastData( const void* data, uint64_t size );
  [void eq::Node::sendData( const void* data, uint64_t size );]
  const void* eq::Node::receiveData( uint64_t* size );
  const void* eq::Node::tryReceiveData( uint64_t* size );
  bool  eq::Node::hasData() const;

  virtual bool Config::startInit( const uint32_t initID );
  virtual bool Config::finishInit();
</pre><!-- */ -->

<h2>Restrictions</h2>
<p>
  Sending data is only supported from the application node. If a reasonable use
  case for sending data from a render node is presented, this can be relaxed.
</p>

<h2>Open Issues</h2>
<p>
  Late inits will require resend of the data.
</p>


#include "footer.shtml"
<!-- $Id$ -->