Copy the KDE 3.5 branch to branches/trinity for new KDE 3.5 features.

BUG:215923


git-svn-id: svn://anonsvn.kde.org/home/kde/branches/trinity/kdemultimedia@1054174 283d02a7-25f6-0310-bc7c-ecb5cbfe19da

1 files changed, 261 insertions, 0 deletions

diff --git a/arts/midi/README.midi b/arts/midi/README.midi
new file mode 100644
index 00000000..e341ca01
--- /dev/null
+++ b/arts/midi/README.midi
@@ -0,0 +1,261 @@
+Midi, Audio and Synchronization:
+================================
+
+1. Introduction
+2. The midi manager
+3. Midi synchronization
+4. Audio timestamping and synchronization
+5. Example code
+
+
+  1. Introduction
+  ---------------
+
+Since aRts-1.0 (as shipped with KDE3.0), aRts provides a lot more
+infrastructure to deal with midi, audio, and their synchronization. The main
+goal is to provide a unified interface between sequencers (or other programs
+that require notes or audio tracks to be played at certain given time stamps)
+and underlying software/hardware that can play notes/audio tracks.
+
+Currently, there exist five distinct destinations that aRts supports, which
+can all be used at the same time or individually, that is:
+
+ * aRts synthetic midi instruments
+ * ALSA-0.5
+ * ALSA-0.9
+ * OSS
+ * other aRts modules (including but not limited to the playback/recording
+   of audio tracks)
+
+  2. The midi manager
+  -------------------
+
+The midi manager is the basic component that connects between applications
+that supply/record midi data, and devices that process midi data. Devices
+might be both, virtual (as in software synthesis) or real (as in hardware
+devices).
+
+From the view of the midi manager, all event streams correspond to one midi
+client. So, a midi client might be an application (such as a sequencer) that
+provides events, or an ALSA hardware device that consumes events. If there
+are multiple event streams, they correspond to multiple clients. That is,
+if an application wishes to play three different midi tracks, one over ALSA,
+and two over two different synthetic instruments, it needs to register itself
+three times, with three different clients.
+
+The midi managers job is to connect midi clients (as in event streams). It
+maintains a list of connections that the user can modify with an application
+like artscontrol. Applications could also, if they wish so, modify this
+connection list. 
+
+As a use case, we'll consider the following: you want to write a sequencer
+application that plays back two different tracks to two different devices.
+You want the user to be able to select these devices in a drop down box for
+each track.
+
+1) getting a list of choices:
+
+First, you will want to obtain a list of choices which the user could possibly
+connect your tracks to. You do so by reading the
+
+interface MidiManager { // SINGLETON: Arts_MidiManager
+    /**
+     * a list of clients
+     */
+    readonly attribute sequence<MidiClientInfo> clients;
+
+	//...
+};
+
+attribute. The three fields of each client that are interesting for you are
+
+struct MidiClientInfo {
+	long ID;
+
+	//...
+
+	MidiClientDirection direction;
+	MidiClientType type;
+	string title;
+};
+
+You would list those devices in the dropdown box that are of the appropriate
+direction, which is mcdRecord, as you would want a client that receives midi
+events (this might be confusing, but you look from the view of the client).
+
+Then, there is the type field, which tells you whether the client is a device-
+like thing (like a synthetic instrument), or another application (like another
+application currently recording a track). While it might not be an impossible
+setup that you send events between two applications, usually users will choose
+such clients that have mctDestination as type.
+
+Finally, you can list the titles in a drop down box, and keep the ID for making
+a connection later.
+
+2) registering clients:
+
+You will need to register one client for each track. Use
+
+	/**
+	 * add a client
+     *
+	 * this creates a new MidiManagerClient
+	 */ 
+	MidiClient addClient(MidiClientDirection direction, MidiClientType type,
+							string title, string autoRestoreID);
+
+to do so.
+
+3) connecting:
+
+As you probably don't want your sequencer user to use artscontrol to setup
+connections between your tracks and the devices, you will need to connect
+your clients to the hardware devices for playing something.
+
+You can connect clients to their appropriate destinations using
+
+	/**
+	 * connect two clients
+	 */
+	void connect(long clientID, long destinationID);
+
+and
+
+	/**
+	 * disconnect two clients
+	 */
+	void disconnect(long clientID, long destinationID);
+
+Keep in mind that a client might be connected to more than one destination
+at the same time, so that you will need to disconnect the old destination
+before connecting the new one.
+
+4) playing events:
+
+You can now play events to the tracks, using each client's
+
+	MidiPort addOutputPort();
+
+function for getting a port where you can send events to. However, you will
+also need to ensure that the events will get synchronized as soon as you are
+playing back events to different devices. Read the next section for details
+on this.
+
+  3. Midi synchronization
+  -----------------------
+
+As soon as you are writing a real sequencer, you might want to output to more
+than one midi device at a time. For instance, you might want to let some of
+your midi events be played by aRts synthesis, while others should be sent
+over the external midi port.
+
+To support this setup, a new interface called MidiSyncGroup has been added. To
+output midi events synchronized over more than one port, you proceed as follows:
+
+a) you obtain a reference to the midi manager object
+
+   MidiManager midiManager = DynamicCast(Reference("global:Arts_MidiManager"));
+   if(midiManager.isNull()) arts_fatal("midimanager is null");
+
+b) you create a midi synchronization group which will ensure that the
+   timestamps of your midi events will be synchronized
+   
+   MidiSyncGroup syncGroup = midiManager.addSyncGroup();
+
+c) you add a client to the midi manager for each port you want to output
+   midi data over
+
+   MidiClient client = midiManager.addClient(mcdPlay, mctApplication, "midisynctest", "midisynctest");
+   MidiClient client2 = midiManager.addClient(mcdPlay, mctApplication, "midisynctest2", "midisynctest2");
+
+d) you insert the clients in the synchronization group
+
+   syncGroup.addClient(client);
+   syncGroup.addClient(client2);
+
+e) you create ports for each client as usual
+
+   MidiPort port = client.addOutputPort();
+   MidiPort port2 = client2.addOutputPort();
+
+f) at this point, you will need to ensure that the midi clients you created
+   are connected, you can either leave the user with artscontrol for doing
+   this, or use the clients and connect methods of the midiManager object
+   yourself (see use case discussed in previous section)
+
+g) you output events over the ports as usual
+
+   /* where t is a suitable TimeStamp */
+   MidiEvent e = MidiEvent(t,MidiCommand(mcsNoteOn|0, notes[np], 100));
+   port.processEvent(e);
+   port2.processEvent(e);
+
+  4. Audio timestamping and synchronization
+  -----------------------------------------
+
+Audio in aRts is usually handled as structures consisting of small modules
+that do something. While this model allows you to describe anything you want
+to, from playing a sample to playing a synthetic sequence of notes with a
+synthetic instruments, it doesn't give you any notion of time. More so, if
+you build a large graph of objects, you might need quite some time for this,
+and you will want to have them all started at the same time.
+
+To solve this issue, an AudioSync interface has been introduced, that allows
+you to start() and stop() either synchronized at a specific point in time.
+
+Suppose you have two synthesis modules which together play back a sample.
+What can you do to start them at the same time?
+
+	Synth_PLAY_WAV wav = //... create on server
+	Synth_AMAN_PLAY sap //... create on server
+	AudioSync audioSync = //... create on server
+
+	wav.filename("/opt/kde3/share/sounds/pop.wav");
+	sap.title("midisynctest2");
+	sap.autoRestoreID("midisynctest2");
+	connect(wav,sap);
+
+	// this queues back start() to be called atomically later
+	audioSync.queueStart(wav);
+	audioSync.queueStart(sap);
+
+	// this line is a synchronized version of
+	// wav.start();
+	// sap.start();
+	audioSync.execute();
+
+You could also play them back at a specific time in the future and query the
+current time using the time and executeAt methods:
+
+interface AudioSync {
+	/**
+	 * the current time
+	 */
+	readonly attribute TimeStamp time;
+
+	//...
+
+	/**
+	 * atomically executes all queued modifications to the flow system
+	 * at a given time
+	 */
+	void executeAt(TimeStamp timeStamp);
+};
+
+Finally, to get synchronized midi and audio, you can insert the AudioSync
+object into a midi synchronization group, then their timestamps will be
+synchronized to those of the midi channels.
+
+  5. Example code
+  ---------------
+
+An example that illustrates most things discussed in this document is
+midisynctest.cc, which plays back two synchronized midi streams and samples.
+Note that you might want to change the source code, as it hardcodes the
+location of the .wav file.
+
+
+Questions and comments are welcome.
+
+Stefan Westerfeld
+stefan@space.twc.de