Media Kit Basics: Consumers and Producers
By Owen Smith, DTS Engineer (orpheus@be.com)


Several weeks ago, Sheppy gave us a starting point for
learning how to use the Media Kit:

http://www.be.com/aboutbe/benewsletter/volume_II/
Issue45.html#Workshop

For many of you, however, that's not nearly enough. You want
to see the creatures that lurk under BSoundPlayer's tame
surface. You want to blast buffers over your media net like
terrified badminton shuttlecocks. You want to run your
fingers through the fertile Media Kit soil. And you need the
whole system at your fingertips, yesterday.

Suits me fine. Here's some code for you to peruse:

ftp://ftp.be.com/pub/samples/r4/media_kit/SoundCapture.zip

This app does two things. First, it records sounds, using an
optional voice-activation feature, and saves the sounds as
raw audio files. Second, it loads and plays back raw audio
sounds so you can audition what you recorded.

Straightforward this app is, but simple it ain't. It would
take me much longer than a newsletter article to explain
its operation from the ground up. So, before you read on,
please familiarize yourself with the basic concepts of
the Media Kit by browsing the freshly painted Be Book:

http://www.be.com/documentation/be_book/index.html

For now, you'll want to concentrate on the Media Kit classes
BMediaNode, BBufferConsumer, BBufferProducer, and
BMediaRoster. You might also glance at the header file
MediaDefs.h, which describes many of the lower-level
structures and defines that we use. Then, look at this
article and the sample code to see how it all fits
together!

The code is pretty heavily commented, even for a DTS
project, so I'll spare the details and just touch on a few
of the more important issues here. OK -- let's get down and
dirty with the Media Kit...

Under the Hood

There are three key classes that make SoundCapture tick:
SoundConsumer, SoundProducer, and CaptureWindow.

SoundConsumer

SoundConsumer is a franchise of BBufferConsumer; its job
is to process (i.e., "consume") buffers that are sent to it
from a higher source. We've tried to make this class
reusable by making it reasonably extensible. In a similar
manner to BSoundPlayer, we do this by defining two places
where you, the SoundConsumer client, need to decide what to
do:

* The Process function, which is called each time a buffer
  of data arrives. You provide functionality to tell it what
  to do with these buffers.

* The Notify function, which is called each time something
  important happens to the node (it starts, stops, or dies
  an abrupt death, for example). You provide functionality
  to dispatch any of these events as you see fit.

You fill in these slots by either subclassing SoundConsumer
and overriding the function, or simply by passing
SoundConsumer a hook function to call instead.

We use the SoundConsumer as a simple recorder. We implement
a Process hook function that writes the buffer's data to
disk, and a Notify hook function to makes sure that things
get cleaned up when the node's about to stop or die.

SoundProducer

SoundProducer is a subsidiary of BBufferProducer; its job is
to produce buffers like clockwork and pass them along to
someone else. Like SoundConsumer, we provide two places for
you to specialize SoundProducer's behavior: Process, which
is called each time a buffer needs to be filled with data;
and Notify, which is called when certain events happen.

We use the SoundProducer to implement simple playback from a
sound file. In this sense, it functions almost exactly like
BSoundPlayer and BSound (save that it only reads raw audio,
doesn't handle multiple sounds, and doesn't do sample format
conversion).

CaptureWindow

In addition to fulfilling its traditional duties as a
BWindow, CaptureWindow also functions in SoundCapture as the
context in which SoundConsumer and SoundProducer are used.
CaptureWindow contains an instance of each class, and does
the following:

* When you click on the Record button, CaptureWindow
  connects its SoundConsumer to the system's audio input
  and starts the node.

* Similarly, when you click on the Play button,
  CaptureWindow connects its SoundProducer to one of the
  system's mixer inputs and starts the node.

* Finally, when you click on Stop (or when one of the
  Process or Notify hook functions has determined that the
  node is done recording or playing), CaptureWindow figures
  out which node is running, stops the node, and disconnects
  it.

Consumer vs. Producer

Looking at the responsibilities of a BBufferConsumer and a
BBufferProducer, you'll notice that there are a lot of
similarities. Here some of the more important ones:

* Both consumers and producers create a port, called the
  Control Port, that the media server uses to send messages
  to them. Messages range from performance event (Start,
  Stop, and Seek) requests, to receiving buffers, to
  messages that you define and send yourself.

* Both create a thread, lovingly referred to as the Big Bad
  Service Thread, which is responsible for handling messages
  sent to the Control Port in a timely manner. Among the
  other things that this thread might do, its primary
  responsibility is to read from the Control Port until a
  message is received, and then handle the message.

* Both override BMediaNode functions that implement certain
  important performance events. Start tells you when your
  node needs to start. Stop tells you when your node needs
  to stop. Finally, Seek tells you when you need to change
  your media time, and what the new media time should be.
  (More on this in a bit.)

At the same time, there are several significant differences
that you should be aware of. Here's a blow-by-blow:

Performance Events and Media Time

Your node can interpret Start, Stop, and Seek events in
various ways, depending on what makes sense for your node.

For SoundProducer, Start and Stop are interpreted to mean
"start producing buffers" and "stop producing buffers."
Our application does not currently define the concept of
media time for its sound producer, so SoundProducer::Seek
has no effect. In the future, media time would probably be
interpreted as the offset in the sound file you're playing.

On the other hand, SoundConsumer currently defines no
behavior for Start and Stop, and simply accepts buffers at
any time. It also doesn't do anything meaningful with Seek
requests, but passes the media time as a timestamp to its
Process hook function, in case that time has any meaning
to SoundConsumer's client.

Timing and SoundConsumer

As the BeOS media system generally runs in real time, timing
issues are perhaps the most critical part of developing a
media node, and are often the trickiest part to get right.

SoundConsumer has it easy, since all it needs to do is grab
buffers as they arrive and blast them to disk. It doesn't
even care whether the buffers were on time or not. So, all
it needs to do is sit around in its Big Bad Service Thread,
waiting patiently for those buffers to arrive. Once messages
do arrive, it handles them immediately.

Because it doesn't really care about Start, Stop, or Seek
requests, SoundConsumer "handles" them instantaneously,
instead of queuing them up for handling at a particular
time. More complicated consumers might need to handle
performance events accurately, and might need to determine
whether incoming buffers are running behind.

Timing and SoundProducer

SoundProducer, on the other hand, is a lot more complicated.
Not only does it need to handle performance events at their
correct times, but it also needs to produce a steady stream
of buffers -- and those buffers have to be sent at precisely
the right time, so that they don't reach the final output
(your headphones) too late, or so early that the latency of
the system is more than it needs to be. Its Big Bad Service
Thread, therefore, does three different things:

* It checks to see if any pending performance events need
  to be handled, and handles them when they do.

* It checks to see if any messages have arrived at the
  Control Port, and handles them when they do.

* When it's time to produce a buffer, SoundProducer stuffs
  a buffer (using the Process function) and sends it off.

One of the keys to understanding the timing of SoundProducer
is the timeout value passed to read_port_etc. This value
determines how long the thread waits in each iteration of
the loop for messages to arrive. This timeout is set to the
time until the next performance event needs to be handled,
or until the next buffer needs to be produced, whichever
comes first. So, this call to read_port_etc really serves
the dual purpose of receiving messages and snoozing until
the next thing needs to happen!

The other key to understanding SoundProducer timing is the
value returned by BTimeSource::RealTimeFor(). This somewhat
misnamed function does not give the absolute real time that
corresponds to a given performance time, but rather gives
you the real time that you need to do things in order for
their effects to take place at the performance time. It does
this by taking into account the latency that you give it --
that is to say, you tell it the difference between the time
at which you decide to do something, and the time at which
that something actually takes effect. The greater your
latency, the earlier you must start things for them to
happen on time. And, as I have been reminded on any number
of occasions, being on time is extremely important.

Parting Thoughts

There are several directions in which this app can grow.
In particular, you could extend either SoundConsumer or
SoundProducer to do all sorts of wild stuff. You could
override the consumer's hook functions to provide, for
example, an oscilloscope node, or a node that performs
spectral analysis. You could also override the
producer's hook functions to perform sound synthesis.
Go nuts!



