This page is under heavy revision. Please refer to
http://doc.desktopgrid.hu/
for up-to-date information and documentation.
The DC-API

Introduction

The DC-API was created by MTA SZTAKI to allow easy implementation and deployment of distributed applications on multiple grid environments.

In order to accommodate the needs of very different grid environments, the DC-API supports only a restricted master-worker programming model. The restrictions include:

  • Master-worker concept: there is a designated master process running somewhere on the grid infrastructure. The master process can submit worker processes called work units.

  • Every work unit is a sequential application.

  • There is support for limited messaging between the master and the running work units. It can be used to send status and control messages, but it is not suitable for parallel programming.

  • There can not be any direct communication between work units.

General concepts

Programming model

DC-API applications consist of two major components: a master application and one or more client applications. The master is responsible for dividing the global input data into smaller chunks and distributing these chunks in the form of work units. Interpreting the output generated by the work units and combining them to form a global output is also the job of the master.

The master application usually runs as a daemon, but it is also possible to write a master that runs periodically (e.g. from cron), processes the outstanding events, and exits.

Client applications are simple sequential programs that take their input from the master, perform some computation on it and produce some output.

Writing a master application

A typical master application does the following steps:

Writing a client application

A typical client application performs the following steps:

Messaging

The DC-API provides limited messaging functionality between the master application and the clients. The DC-API has the following features and restrictions:

  • Messages are not reliable in the sense that if the client is not actually running when a message is being sent to it (e.g. because it is queued by the backend grid infrastructure), then the message may be silently dropped.

  • The ordering of messages is not necessarily maintained.

  • Messages are delivered asynchronously. There is no limit for the time elapsed before a message is actually delivered.

Due to the above restrictions, DC-API messages are not suitable for message-based parallel processing. They are meant for sending short status messages about long-running operations, or for sending control messages like a command to cancel a given computation.