General concepts

Programming model

DC-API applications consist of two major components: a master application and one or more client applications. The master is responsible for dividing the global input data into smaller chunks and distributing these chunks in the form of work units. Interpreting the output generated by the work units and combining them to form a global output is also the job of the master.

The master application usually runs as a daemon, but it is also possible to write a master that runs periodically (e.g. from cron), processes the outstanding events, and exits.

Client applications are simple sequential programs that take their input from the master, perform some computation on it and produce some output.

Writing a master application

A typical master application does the following steps:

  • Initializes the DC-API library by calling DC_initMaster() function.

  • Calls the DC_setResultCb() function and optionally some of the DC_setSubresultCb(), DC_setMessageCb(), DC_setSuspendCb() and DC_setValidateCb() functions, depending on the features (messaging, subresults etc.) it wants to use.

  • In its main loop, the master calls the DC_createWU() function to create new work units when needed. If the total number of work units is small (depending on the grid infrastructure), then the master may also create all the work units in advance. If the total number of work units is too large for this, the master may use the DC_getWUNumber() function to determine the number of running work units, and create new work units only if this number falls below a certain threshold.

  • Also in its main loop the master calls the DC_processMasterEvents() function that checks for outstanding events and invokes the appropriate callbacks.

    Alternatively, the master may use the DC_waitMasterEvent() and DC_waitWUEvent() functions instead of DC_processMasterEvents() if it prefers to receive event structures instead of using callbacks.

Writing a client application

A typical client application performs the following steps:

  • Initializes the DC-API library by calling DC_initClient() function.

  • Identifies the location of its input/output files by calling the DC_resolveFileName() function.

    Note

    The client application may not assume that it can read/create/write any files other than the names returned by DC_resolveFileName().

  • During the computation, the client should periodically call the DC_checkClientEvent() function and process the received events.

  • If possible, the client should call the DC_fractionDone() function with the fraction of the work completed. On some grid infrastructures (e.g. BOINC) this will allow the client's supervisor process to show the progression of the application to the user.

    Ideally the value passed to the DC_fractionDone() function should be proportional to the time elapsed so far compared to the total time that will be needed to complete the computation.

  • The client should call the DC_finishClient() function at the end of the computation. As a result all output files will be sent to the master and the master will be notified about the completion of the work unit.

Messaging

The DC-API provides limited messaging functionality between the master application and the clients. The DC-API has the following features and restrictions:

  • Messages are not reliable in the sense that if the client is not actually running when a message is being sent to it (e.g. because it is queued by the backend grid infrastructure), then the message may be silently dropped.

  • The ordering of messages is not neccessarily maintained.

  • Messages are delivered asynchronously. There is no limit for the time elapsed before a message is actually delivered.

Due to the above restrictions, DC-API messages are not suitable for message-based parallel processing. They are meant for sending short status messages about long-running operations, or for sending control messages like a command to cancel a given computation.