14.1. Introducing aRts/MCOP

The road for KDE 2.0 (and later versions) is integration through one consistent streaming-media technology. The idea is that you can write any multimedia task as little pieces, which pass multimedia streams.

Quite some time before KDE 2.0, I first heard of the plans of KAudioServer2 (from Christian Esken), which was an attempt to improve and rewrite the audioserver to support streaming media to a certain degree. On the other hand, I had been working on aRts (analog, real-time synthesis) software for quite a while and had already implemented some nice streaming support. In fact, aRts was a modular software synthesizer that worked through little plug-ins and streams between them. And, most important of all, aRts was already working great.

So, after some considerations, we decided at the KDE 2.0 Developer meeting to make aRts the base for all streaming multimedia under KDE. Many things would have to be changed to come from one synthesizer to a base for all multimedia tasks, but it was the much better approach than trying to do something completely new and different, because aRts was already proven to work.

As I see it, the important parts of streaming multimedia support are

• An easy way to write small modules, which can be used for streaming (plug-ins).

• A way to define how these modules communicate (what types of data they accept, what properties they have, what functions they support).

• A scheduler that decides what module gets executed when—this is necessary because you usually have lots of small modules running in one task.

• A transfer layer, which ensures that modules running in different processes/applications or on different computers can communicate.

How these things work is probably illustrated best with a small example. Assume you want to listen to a beep while the left speaker should be playing a 440Hz frequency and the right speaker is playing a 880Hz frequency. That would look something like the following:

As you see, the task has been divided into very small components, each of which do only a part of the whole. The frequency generators only generate the frequency (they can also be used for other wave forms), nothing more. The sine wave objects only calculate the sinus of the values they get. The play object only takes care that these things really reach your sound card. To get a first impression, the source code for this example is shown in Listing 14.1:

   1 
   2  1: // first_example.cc
   3  2:
   4  3: #include "artsflow.h"
   5  4: #include "connect.h"
   6  5:
   7  6: using namespace Arts;
   8  7:
   9  8: int main()
  10  9: {
  11 10:     Dispatcher dispatcher;
  12 11:
  13 12:     Synth_FREQUENCY freq1,freq2;   // object creation
  14 13:     Synth_WAVE_SIN  sin1,sin2;
  15 14:     Synth_PLAY      play;
  16 15:
  17 16:     setValue(freq1, 440.0);       // set frequencies
  18 17:     setValue(freq2, 880.0);
  19 18:
  20 19:     connect(freq1, sin1);         // object connection
  21 20:     connect(freq2, sin2);
  22 21:     connect(sin1, play, "invalue_left");
  23 22:     connect(sin2, play, "invalue_right");
  24 23:
  25 24:     freq1.start(); freq2.start();  // start&go
  26 25:     sin1.start(); sin2.start();
  27 26:     play.start();
  28 27:     dispatcher.run();
  29 28: }
  30 

Now, while you're thinking of that simplistic example, consider Figure 14.2:

Figure 14.2 illustrates a real-life example. I've simply composed three tasks done at the same time.

First, consider the MIDI player. The MIDI-player component is probably reading a file and sending out MIDI events. These are sent through a software synthesizer, which takes the incoming MIDI events and converts them to an audio stream. This is not about your hardware wave table on the sound card; all things that we are talking about here are happening before the data is sent to the sound card.

On the other hand, there is the game. Games often have very specific requirements for how they calculate their sound, so they might have a complete engine that does this task. One example is Quake. It calculates sound effects according to the player's position, so you can orient yourself by listening closely to what you hear. In that case, the game generates a complete audio stream itself, which only will be sent to the mixer.

The next chain is the one with the microphone attached. The microphone output is sent through a pitch-shifting effect in this example. Then the output goes through the mixer, the same as everything else. Through the pitch shifting, your voice sounds higher (or lower) because the frequency changes give this a funny cartoon-character effect. If you like, you can also imagine a more "serious" application, such as speech recognition or Internet telephony at this place.

Finally, everything is mixed in the mixer component, and then, after sending it through a last effect (which adds the reverb effect), played over your sound card.

This example shows a bit more of what the multimedia support does here. You, for instance, see that not all components that are involved are in the same process. You wouldn't want to run your Quake game inside the audioserver, which also does the other tasks. Maybe your MIDI player is also external; maybe it is a component that runs inside the audioserver. Thus, the signal flow is distributed between the processes. The components that are responsible for certain tasks run where it fits best.

You also see that different kinds of streams are involved. The first is normal audio streams, which are managed nicely by the aRts/MCOP combination (and the most convenient method). The second is the MIDI stream. These differ a lot. An audio stream always carries data. In one second, 44,100 values are passed across the stream. In contrast, a MIDI stream transmits something only when it is needed. When a note is played, a small event is sent over the stream; when nothing happens, nothing is sent.

The third type is byte audio, which refers to the way the game in that case could produce audio. Byte stream is the same format that would normally be replayed through the sound card (16 bit, little endian, 44kHz, stereo). To process such data with the mixer, it needs to go through a converter because the mixer only mixes "real" audio streams.