This section describes some of the &arts; work that is in progress. Development progresses quickly, so this information may be out of date. You should check the TODO list file and the mailing list archives to see what new functionality is planned. Feel free to get involved in new design and implementation. This is a draft document which tries to give you an overview how new technologies will be integrated in &arts;. Namely, it does cover the following: How interfaces work. Codecs - decoding of mp3 or wav streams in a form that they can be used as data. Video. Threading. Synchronization. Dynamic expansion/masquerading. Dynamic composition. &GUI; &MIDI; This is work in progress. However, it should be the base if you want to see new technology in &arts;. It should give you a general idea how these problems will be addressed. However, feel free to correct anything you see here. Things that will be use &arts; technology (so please, coordinate your efforts): KPhone (voice over IP) &noatun; (video / audio player) &artscontrol; (sound server control program, for scopes) Brahms (music sequencer) Kaiman (&kde;2 media player - kmedia2 compliant) mpglib/kmpg (mpg audio and video playing technology) SDL (direct media layer for games not yet started but maybe nice) electric ears (author contacted me - status unknown) &MCOP; interfaces are the base of the &arts; concept. They are the network transparent equivalent to C++ classes. Whenever possible you should orient your design towards interfaces. Interfaces consist of four parts: Synchronous streams Asynchronous streams Methods Attributes These can be mixed in any way you like. New technologies should be defined in terms of interfaces. Read the sections about asynchronous streams and synchronous streams, as well as the KMedia2 interfaces, which are a good example how such things work Interfaces are specified in .idl code and run through the mcopidl compiler. You derive the Interfacename_impl class to implement them, and use REGISTER_IMPLEMENTATION(Interfacename_impl) to insert your object implementations into the &MCOP; object system. The kmedia2 interfaces allow you to ignore that wav files, mp3s and whatever consist of data streams. Instead, you only implement methods to play them. Thus, you can write a wave loading routine in a way that you can play wave files (as PlayObject), but nobody else can use your code. Asynchronous streams would be the alternative. You define an interface which allows you to pass data blocks in, and get data blocks out. This looks like that in &MCOP;: interface Codec { in async byte stream indata; out async byte stream outdata; }; Of course codecs could also provide attributes to emit additional data, such as format information. interface ByteAudioCodec { in async byte stream indata; out async byte stream outdata; readonly attribute samplingRate, bits, channels; }; This ByteAudioCodec for instance could be connected to a ByteStreamToAudio object, to make real float audio. Of course, other Codec types could involve directly emitting video data, such as interface VideoCodec { in async byte stream indata; out video stream outdata; /* note: video streams do not exist yet */ }; Most likely, a codec concept should be employed rather than the you know how to play and I don't way for instance WavPlayObject currently uses. However, somebody needs to sit down and do some experiments before an API can be finalized. My idea is to provide video as asynchronous streams of some native &MCOP; data type which contains images. This data type is to be created yet. Doing so, plugins which deal with video images could be connected the same way audio plugins can be connected. There are a few things that are important not to leave out, namely: There are RGB and YUV colorspaces. The format should be somehow tagged to the stream. Synchronization is important. My idea is to leave it possible to reimplement the VideoFrame class so that it can store stuff in a shared memory segment. Doing so, even video streaming between different processes would be possible without too much pain. However, the standard situation for video is that things are in the same process, from the decoding to the rendering. I have done a prototypic video streaming implementation, which you can download here . This would need to be integrated into &MCOP; after some experiments. A rendering component should be provided that supports XMITSHM (with RGB and YUV), Martin Vogt told me he is working on such a thing. Currently, &MCOP; is all single threaded. Maybe for video we will no longer be able to get around threading. Ok. There are a few things that should be treated carefully: SmartWrappers - they are not threadsafe due to non-safe reference counting and similar. Dispatcher / I/O - also not threadsafe. However, what I could imagine is to make selected modules threadsafe, for both, synchronous and asynchronous streaming. That way - with a thread aware flow system, you could schedule the signal flow over two or more processors. This would also help audio a lot on multiprocessor things. How it would work: The Flow System decides which modules should calculate what - that is: video frames (with process_indata method) synchronous audio streams (calculateBlock) other asynchronous streams, mainly byte streams Modules can calculate these things in own threads. For audio, it makes sense to reuse threads (⪚ render on four threads for four processors, no matter if 100 modules are running). For video and byte decompression, it may be more confortable to have a blocking implementation in an own thread, which is synchronized against the rest of &MCOP; by the flow system. Modules may not use &MCOP; functionality (such as remote invocations) during threaded operation. Video and &MIDI; (and audio) may require synchronization. Basically, that is timestamping. The idea I have is to attach timestamps to asynchronous streams, by adding one timestamp to each packet. If you send two video frames, simply make it two packets (they are large anyway), so that you can have two different time stamps. Audio should implicitely have time stamps, as it is synchronous. It should be possible to say: An effect FX is composed out of these simpler modules. FX should look like a normal &MCOP; module (see masquerading), but in fact consist of other modules. This is required for &arts-builder;. All &GUI; components will be &MCOP; modules. They should have attributes like size, label, color, ... . A RAD builder (&arts-builder;) should be able to compose them visually. The &GUI; should be saveable by saving the attributes. The &MIDI; stuff will be implemented as asynchronous streams. There are two options, one is using normal &MCOP; structures to define the types and the other is to introduce yet another custom types. I think normal structures may be enough, that is something like: struct MidiEvent { byte b1,b2,b3; sequence<byte> sysex; } Asynchronous streams should support custom stream types.