Application Frameworks

Syzygy programs are based on a framework or application object that manages the many tasks of a networked VR application. The framework object is responsible for program launching and shutdown. It also handles communications, rendering synchronization, graphics, sound, and input-event handling. The framework supports communication across different computer architectures and operating systems, so you can mix and match computers at will in your cluster. For example, a cluster could be composed of both big-endian and little-endian machines.

The only currently-supported application framework is the master/slave. In this framework the programmer is in charge of sharing data between instances of the application so it requires some care to produce an application that displays consistently on all cluster render nodes. On the In a program based on the arMasterSlaveFramework class, a copy of the application will run on each render node. We refer to such applications as master/slave because one instance of the application, the master, controls the operation of the others, the slaves. Rendering is done with OpenGL commands written by the application programmer, although the framework handles the computation of the projection matrix and utility classes and functions are available for loading certain kinds of content (e.g. texture images and 3-D models in obj format). The master/slave framework thus offers an easy migration path for existing applications to cluster-based operation; two of the included sample applications, "atlantis" and "coaster", were easily ported to Syzygy from the GLUT distribution using the master/slave framework.

The master/slave framework class is a subclass of arSZGAppFramework.

Samples can be found under szg/src/demo. Please see the examples chapter of this documentation for more information.

Framework Features

The framework has routines for setting the unit conversion factors for both rendering and sound production (both default to 1, i.e. default units are feet:

virtual arSZGAppFramework::void setUnitConversion( float program_units_per_foot );
virtual arSZGAppFramework::void setUnitSoundConversion( float program_units_per_foot );

Note that if the unit conversion factors are set, it should be done before the framework's init() method is called, because they may be required when reading in configuration database parameters are furing init().

Routines for setting the user's interocular separation and the near and far clipping planes:

void arSZGAppFramework::setEyeSpacing( float feet );
void arSZGAppFramework::setClipPlanes( float near, float far );

The clipping planes are always set by the application programmer, using the units specified by the unit conversion factors. The eye spacing, on the other hand, is read from the Syzygy database; if you require to set it in code, do it after the framework's init(), because otherwise it will be overridden by the value from the database.

Accessing data in Cluster Mode

Previously, program data had to be located in a subdirectory of a directory on the Syzygy data path (specified by the database variable SZG_DATA/path, see Syzygy Resource Path Configuration). This path is accessed using this method:

  const string arSZGAppFramework::getDataPath()

and the path of a file in the my_app subdirectory of a directory on this path can be found using:

string ar_fileFind( <fileName>, 'my_app', framework.getDataPath() );

Starting with Syzygy 1.1, most data files can be placed in the same directory as the program (again, see Syzygy Resource Path Configuration) and read using normal file-access methods with application-relative paths. For example, a file 'foo.txt' in the same directory as the application can be read using FILE* f = fopen('foo.txt','r'), even in cluster mode.

Things get a bit trickier when it comes to files that have to be opened by a Syzygy service. In a master/slave program only sound files fall into this category (they must be read and played by SoundRender); in a distributed scene graph program, any data file such as a texture map or a .obj model must be read by szgrender.

These files can still be placed with the application, but the application must then tell Syzygy where to find them. It does so using the frameworks' setDataBundlePath() method, passing in the group name of a Syzygy path variable and a subdirectory name.

Two examples:

  1. A C++ app, located in subdirectory 'my_app' in directory 'apps', which in turn is part of the SZG_EXEC/path variable. A set of sound files are also in the 'my_app' directory. Calling
      framework.setDataBundlePath( "SZG_EXEC", "my_app" );
    
    will allow SoundRender to find the files.

  2. A Python program, located in a subdirectory 'my_app' of directory 'apps', which is listed in the Python search path SZG_PYTHON/path. The appropriate call would be:
      framework.setDataBundlePath( "SZG_PYTHON", "my_app" )
    

Input Events

The framework provides methods for polling the current state of the various input events. Input events are distinguished by type (button,axis, or matrix) and index (a non-negative or unsigned integer). By convention, the head position and orientation are assumed to be contained in matrix event 0, the hand placement in matrix #1, axis 0 is assumed to represent left/right movement of a joystick and axis 1 front/back joystick motion for driving.

int arSZGAppFramework::getButton( unsigned int index );

Returns the current state (0 or 1) of the named button. Button numbering starts at 0. However, the following to routines are more commonly used:

int arSZGAppFramework::getOnButton( unsigned int index );
int arSZGAppFramework::getOffButton( unsigned int index );

These return 0 or 1 depending on whether the specified button has been pressed or released since the previous frame.

float arSZGAppFramework::getAxis( unsigned int index );

Returns the value of the named axis. Axis events representing joysticks are assumed to lie between -1.0 and 1.0.

arMatrix4 arSZGAppFramework::getMatrix( unsigned int index );

Returns the value of the named matrix. Information about the arMatrix4 object can be found by examining szg/src/math/arMath.h.

The framework is capable of handling unlimited event indices, and also has methods for determining the number of event indices available:

unsigned int arSZGAppFramework::getNumberButtons();
unsigned int arSZGAppFramework::getNumberAxes();
unsigned int arSZGAppFramework::getNumberMatrices();

The framework provides a pointer to the current input state, which is not commonly used but provides more complete access to input state:

const arInputState* getInputState();

The framework supports installation of a single-event callback or filter:

bool arSZGAppFramework::setEventFilter( arFrameworkEventFilter* filter );
void arSZGAppFramework::setEventCallback( 
           bool (*eventCallback)( arSZGAppFramework&, arInputEvent&, arCallbackEventFilter& )
           );

These permit direct access to every single input event immediately as it comes in. They're functionally equivalent, the first is just a bit more object-oriented. Note that they run in a separate thread from the rest of your application, so only use if you're familiar with multi-threaded programming.

The framework supports the Syzygy navigation utilities.

Other Stuff

Speech (Windows only). If the Microsoft Speech API (SAPI) was present during compilation, the framework supports text-to-speech using the following method:

void arSZGAppFramework::speak( const std::string& message );

In Cluster Mode the utterance is performed by SoundRender, so that's the component that needs to be running on Windows for this to work. In Standalone Mode the sound component gets embedded into the application, so it has to be a Windows app. You can control the volume, pitch, etc. using embedded XML tags, see the SAPI documentation.

bool arSZGAppFramework::setInputSimulator( arInputSimulator* sim );

Installs your own version of the Input Simulator.

string arSZGAppFramework::getLabel();

Returns the name of your application.

bool arSZGAppFramework::getStandalone();

Is the program running in standalone (true) or cluster (false) mode?

arHead* arSZGAppFramework::getHead();

Get a pointer to the Head object, which contains information about eye spacing and so on. In a master/slave application it gets shared from master to slaves, so don't change parameters on slaves.

virtual void arSZGAppFramework::setFixedHeadMode(bool isOn) ;

Force fixed-head mode (but only for screens that are configured to allow it). See Graphics Configuration.

virtual arMatrix4 arSZGAppFramework::getMidEyeMatrix();

Return the placement (position+orientation) matrix for the midpoint of the two eyes.

virtual arVector3 arSZGAppFramework::getMidEyePosition();

Return the position of the midpoint of the two eyes.

arAppLauncher* arSZGAppFramework::getAppLauncher();

Get a pointer to the arAppLauncher, which contains information about the virtual computer the application is running on.

arGUIWindowManager* arSZGAppFramework::getWindowManager( void );

Get a pointer to the Syzygy window manager. Sorry, it isn't at all documented yet. You'll have to look at header files to figure out what you can do with it (look in src/graphics at any file whose name begins with "arGUI"...

arSZGClient* arSZGClient::getSZGClient() { return &_SZGClient; }

Get a pointer to the arSZGClient, which allows you to get and set parameters in the Syzygy database and to send messages to other Syzygy components. There are several sections devoted to the arSZGClient in the Distributed Operating System chapter.

Master/Slave Framework

Writing a master/slave program is conceptually similar to writing an OpenGL/GLUT program: Rendering is done by OpenGL calls that you write, and the application framework controls an event loop that calls callback functions that you define. The similarities are not accidental, as this framework was initially based on GLUT. It isn't any more, however.

NOTE: Starting with Syzygy 1.1, the GLUT headers are no longer automatically included in master/slave programs. You can still use a few of the GLUT rendering functions (the ones for drawing objects), but calling any of the other functions, such as those for window creation/manipulation, will crash your program. You will need to include the GLUT header if you want to use e.g. glutSolidCube() and similar functions. This is done differently on different platforms, so we have provided the new header file arGlut.h to handle this for you.

You can use an arMasterSlaveFramework in one of two ways. In the old way, during framework initialization you install callback functions to be called at specific points in the event loop. In the new, more object-oriented way, you create a sub-class of the arMasterSlaveFramework class and in it override the methods that call the callback functions. There is one exception, the single-event callback, which must be handled by installing a function.

The directory szg/skeleton represents a build template for master/slave applications. Copy that entire directory wherever you want (re-name it if you want). See the relevant section of Compiling C++ Programs for more information. szg/skeleton/src contains two files, skeleton.cpp and oopskel.cpp. These do exactly the same thing, but skeleton.cpp does it by installing callbacks and oopskel.cpp does it by sub-classing.

Now we'll list the callbacks roughly in the order in which they are called. The old-style callback function will be listed together with the new-style callback method. Note that in the former case, each callback function's signature includes a reference (fw) to the framework object; in the latter case this is of course not necessary, as the callbacks are methods of the framework.

void arMasterSlaveFramework::setStartCallback(
              bool (*startCallback)(arMasterSlaveFramework& fw,
                                    arSZGClient& client)
              );

virtual bool arMasterSlaveFramework::onStart( arSZGClient& SZGClient );

Called to do application-global initialization. Must not do OpenGL initialization, as it is called before a window is created. If it does not return true, the application will abort.

void arMasterSlaveFramework::setWindowStartGLCallback(
              void (*windowStartGL)( arMasterSlaveFramework&,
                                     arGUIWindowInfo* )
              );

virtual void arMasterSlaveFramework::onWindowStartGL( arGUIWindowInfo* );

This is where you do OpenGL initialization. Called once/window (your app may have multiple windows, this is specified in the Graphics Configuration), immediately after window creation.

void arSZGAppFramework::setEventQueueCallback(
              bool (*eventQueue)( arSZGAppFramework& fw,
                                  arInputEventQueue& theQueue )
              );

virtual void arSZGAppFramework::onProcessEventQueue( arInputEventQueue& theQueue );

Called once/frame only on the master, to process buffered input events. the arInputEventQueue contains all events received since the previous frame. Note that this is only here for completeness; in practice it's always easier to perform the same tasks in the pre-exchange callback below, and if you really need immediate access to particular input events see the section on Advanced Input Event Handling below.

void arMasterSlaveFramework::setPreExchangeCallback(
              void (*preExchange)(arMasterSlaveFramework& fw)
              );

virtual void arMasterSlaveFramework::onPreExchange( void );

Called once/frame only on the master, after buffered input events have been processed and before data are transferred to the slaves. The current input state can be polled using the get<event_type>() methods described above. This is usually where user interaction is handled, after which data are packed into the framework for transfer to slaves.

void arMasterSlaveFramework::setPostExchangeCallback(
              void (*postExchange)(arMasterSlaveFramework& fw)
              );

virtual void arMasterSlaveFramework::onPostExchange( void );

Called once/frame on master and slaves after data transfer from the master. Some additional render-related processing can be done here.

void arMasterSlaveFramework::setWindowCallback(
              void (*windowCallback)( arMasterSlaveFramework& )
              );

virtual void arMasterSlaveFramework::onWindowInit( void );

Prepare the window for rendering. The default behavior is to call the following function (which you can also call):

void ar_defaultWindowInitCallback() {
  glEnable(GL_DEPTH_TEST);
  glColorMask( GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE );
  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
}

void arMasterSlaveFramework::setDrawCallback(
              void (*draw)(arMasterSlaveFramework& fw,
                           arGraphicsWindow& win,
                           arViewport& vp )
              );

virtual void arMasterSlaveFramework::onDraw( arGraphicsWindow& win,
                                             arViewport& vp );

Called possibly multiple times/frame (actually, once/viewport) to draw a viewport. Note that a Syzygy viewport is a bit more than an OpenGL viewport; for example, it includes a specification of color buffers, so anaglyph (red/gree) stereo rendering is one using two Syzygy viewports. So is OpenGL hardware (stereo-buffered) stereo. This of course means that you shouldn't do any computations in this callback, only rendering: Nothing that will change the state of your application.

void arMasterSlaveFramework::setDisconnectDrawCallback( 
              void (*disConnDraw)( arMasterSlaveFramework& )
              );

virtual void arMasterSlaveFramework::onDisconnectDraw( void );

Called to draw the screen once/frame on slaves that are not connected to the master. This is for putting up a splash screen or whatever. Note that the window-init callback is not called, so you need to clear the window yourself.

void arMasterSlaveFramework::setPlayCallback(
              void (*play)(arMasterSlaveFramework& fw)
              );

Not sure when this is called.

void arMasterSlaveFramework::setWindowEventCallback(
              void (*windowEvent)( arMasterSlaveFramework&, arGUIWindowInfo* )
              );

virtual void arMasterSlaveFramework::onWindowEvent( arGUIWindowInfo* );

Called once for each GUI window event (e.g. resizing, dragging, maximizing, etc. The passed structure is defined in szg/src/graphics/arGUIInfo.h.

void arMasterSlaveFramework::setExitCallback(
              void (*cleanup)( arMasterSlaveFramework& )
              );

virtual void arMasterSlaveFramework::onCleanup( void );

Called before the application exits.

void setUserMessageCallback(
        void (*userMessageCallback)( arMasterSlaveFramework&,
                                     const std::string& messageBody )
              );

virtual void arMasterSlaveFramework::onUserMessage(
                                          const string& messageBody );

Called whenever the application receives a user message (sent from e.g. the dmsg commandline program--see the Distributed Operating System chapter. The message_type should be the string 'user').

void arMasterSlaveFramework::setOverlayCallback(
              void (*overlay)( arMasterSlaveFramework& )
              );

virtual void arMasterSlaveFramework::onOverlay( void );

Not sure what this is for.

void arMasterSlaveFramework::setKeyboardCallback(
              void (*keyboard)( arMasterSlaveFramework&, arGUIKeyInfo* )
              );

virtual void arMasterSlaveFramework::onKey( arGUIKeyInfo* );

Called for keypresses when running programs in Standalone Mode. The arGUIKeyInfo structure is defined in szg/src/graphics/arGUIInfo.h.

void arMasterSlaveFramework::setMouseCallback(
              void (*mouse)( arMasterSlaveFramework&, arGUIMouseInfo* )
              );

virtual void arMasterSlaveFramework::onMouse( arGUIMouseInfo* );

Called for mouse movement when running programs in Standalone Mode. The arGUIMouseInfo structure is defined in szg/src/graphics/arGUIInfo.h.

Note that strictly speaking, only the start callback is necessary. In other words, you could write a program that did not install any of the other callbacks and it would compile and, provided your start callback returned true, it would run. It just wouldn't look very interesting, and it might print out a warning on each frame to the effect that you hadn't set a draw() callback. It is up to you to decide which callbacks your application needs.

Sequence of Operations

An arMasterSlaveFramework application begins by calling the init method, passing in the command line parameters:

if (!framework.init(argc,argv))
  return 1;

The application should quit if init fails (returns false). Next, the various callbacks are set, including the important start callback where shared memory is registered. Other application-specific initialization can also occur at this time, but as of Syzygy 0.8, OpenGL initialization should not be done in the start callback! OpenGL initialization must now be done in the windowStartGL callback. The start callback is now called before windows are created, whereas the windowStartGL callback comes after window creation. The old start callback was split like this because now Syzygy applications can open more than one window. The start callback is called once for the entire application and the windowStartGL callback is called once for each window.

Finally, the application should call the start() method to set the application in motion. It first executes the user-defined startCallback(...). If this callback returns false, the start() method returns false. Otherwise, it calls the user-defined windowStartGL() callback once for each graphics window (usually just one). Finally, it begins running an event loop defined by the other callbacks. As with init(...), if start() returns false then the application should terminate.

if (!framework.start())
  return 1;

We now detail the event loop:

  1. Poll input devices: The master application instance is connected to input devices. Here, it copies the current values into memory so they can be exported via the getButton(...), getAxis(...), and getMatrix(...) methods. As it does this, it calls any user-defined event filter or single-event callback installed using setEventCallback() on each event and caches the result. The use of these cached values ensures coherency in applications that depend on input device state in the event loop stages occuring after shared-memory export.

  2. Call the user-defined eventQueue() callback: This is an alternative to the preExchange() callback. It provides the complete queue of events that have arrived since the last frame; otherwise it is functionally identical to preExchange(). Generally speaking, if you need ensure access to every single event, it is easier to use the single-event callback in conjunction with preExchange(), but this callback is provided for those who prefer working with event queues.

  3. Call the user-defined preExchange() callback (master only): Take an action before shared memory is exported from the master application instance to the slave instances. Called only on the master. This is where you would normally put code to handle user interaction and to do state-changing computations.

  4. Shared memory export: Shared memory is exported from the master application instance to the slave application instances. This includes both user-defined blocks of shared-memory and some system level infomation. The system material includes the current time (milliseconds elapsed since initialization on the master) and the time needed to execute the last event loop. It also includes a navigation matrix and the cached input device values.

  5. Call the user-defined postExchange() callback: Take whatever action the user specified. Note that it is safe to query input device values here; the input state is automatically transferred to the slaves during the exchange.

  6. Call the user-defined sound (play()) callback: Play sounds. See the Sound API documentation for examples of how to make sounds.

  7. Call the user-defined draw() callback: Setup the matrix stack using the current head position and information about the screen configuration attached to this pipe. Then execute the user-defined draw callback.

  8. Synchronization: All connected application instances pause here until all are ready. A graphics buffer swap then occurs.

Data Transfer from Master to Slaves

Now, we examine the API in more depth, starting with the way the programmer registers shared memory. In the user-defined initCallback(...), the programmer should register shared memory. There are two kinds of shared memory: application-managed and framework-managed. Application-managed memory is fixed in size, whereas framework-managed is dynamic. The latter can be more convenient, but of course it means that you have to check the size in the slave instances before reading out the data.

Registering application-managed memory is one using the following method of the arMasterSlaveFramework object:

bool arMasterSlaveFramework::addTransferField(string fieldName,
                                              void* memoryPtr,
                                              arDataType theType,
                                              int numElements)

The parameter "fieldName" gives the memory a descriptive name. You pass in an already allocated pointer "memoryPtr" to a block of memory of type given by "theType" and of dimension "numElements". The data type needs to be one of AR_INT, AR_FLOAT, AR_DOUBLE, or AR_CHAR. Note that registering memory is done both in the master instance and the slave instances of the application. Once memory has been registered, the programmer uses the pointer normally, with awareness that the contents of the memory block are transfered from the master to the slaves in step 3 of the event loop.

As an example, the following statement registers a block of 16 floats:

framework.addTransferField("manipulation matrix", void* floatPtr,
                           AR_FLOAT, 16);

Framework-managed shared memory is registered in a similar way:

bool arMasterSlaveFramework::addInternalTransferField(
                                  std::string fieldName,
                                  arDataType dataType,
                                  int numElements );

The pointer argument is omitted, and the numElements argument now denotes the initial size. The size can be changed by calling:

bool arMasterSlaveFramework::setInternalTransferFieldSize(
                                  std::string fieldName,
                                  arDataType dataType,
                                  int newSize );

One then gets a pointer to the memory, on either master or slave, using:

void* arMasterSlaveFramework::getTransferField( std::string fieldName,
                                                arDataType dataType,
                                                int& numElements );

...with the actual size being returned in the numElements parameter, which is a reference.

There is a third, more advanced way to transfer data. Specifically, if you have a variable-sized set of objects of a class that you have defined, you can create an STL-type container for them that is easily synchronized between master and slaves, with objects automatically created and deleted on slaves to mirror the set on the master. See the comments in szg/src/framework/arMSVectorSynchronizer.h for details.

Time

The arMasterSlaveFramework objects also maintain consistent time across nodes. This can be consistently accessed after the shared-memory exchange step of the event loop.

double arMasterSlaveFramework::getTime()

Returns the time in milliseconds that have elapsed on the master since completion of initialization.

double arMasterSlaveFramework::getLastFrameTime()

Returns the time in milliseconds for the last iteration of the event loop. measured from one "poll input devices" step tp the next.

Sometimes it is necessary to determine if one is the master node or not. This is done by the following API call:

bool arMasterSlaveFramework::getMaster()

Returns whether or not this is the master application instance.

As mentioned above, this framework supports the navigation utilities. Any of the routines that modify this navigation may be used, but they should only be called on the master in the preExchange() callback. The framework automatically copies this matrix from the master to each of the slaves. As mentioned in the doc chapter on navigation, the frameworks have two navigation-related methods:

void arMasterSlaveFramework::navUpdate()
void arMasterSlaveFramework::loadNavMatrix()

navUpdate(), like other navigation-matrix modifying routines, should only be called on the master in preExchange(). loadNavMatrix(), which loads the current navigation matrix onto the OpenGL matrix stack, should be called on all instances at the beginning of the drawCallback().

Finally, the arMasterSlaveFramework object includes an internal graphics database that uses the same API as that used in writing distributed scene graph applications. However, in this case, the scene graph database is not shared between master and slaves; each instance of the application has its own independent database. This functionality is included so that programmers can make use of arGraphicsDatabase features, like import filters for 3ds objects. Manipulation of the database can be done using the API outlined in the scene graph documentation chapter. Please note that the "dgSetGraphicsDatabase" command is not necessary in this context. This is automatically done by the framework object. Finally, we outline the one arMasterSlaveFramework method specifically tailored to this:

void arMasterSlaveFramework:draw()
  Draw the internal graphics database.

Finally, it should be possible to integrate master/slave applications with other libraries that themselves seek to control the event loop or on based on graphics system other than OpenGL. To make this possible, the programmer needs to issue the following call instead of start():

bool arMasterSlaveFramework::startWithoutGLUT()

As before, the program should abort if this call returns false. The programmer now has responsibility for calling (or causing to be called) a preDraw() method before each frame is drawn and a postDraw() method after each frame is drawn (but before the buffer swap command has been issued). Methods for retrieving the framework's computed projection and modelview matrices are also provided. This enables the programmer to directly manipulate the viewing API with which he is working.

void arMasterSlaveFramework::preDraw()
  Executes those parts of the event loop that occur before drawing.
void arMasterSlaveFramework::postDraw()
  Executes those parts of the event loop that occur after drawing but
  before buffer swap (really just synchronization).
arMatrix4 arMasterSlaveFramework::getProjectionMatrix()
  Returns the projection matrix calculated by the framework based on
  screen characteristics, head position, and head orientation.
arMatrix4 arMasterSlaveFramework::getModelviewMatrix()
  Returns the modelview matrix calculated by the framework based on
  screen characteristics, head position, and head orientation.