Category Archives: Tutorial

Paper Piano

Paper Piano

You can extend the alligator clip wires by connecting to other conductive materials. Copper tape is a good conductor and is easy to work with.

materials:

– construction paper

– copper tape

– Sound Clippys Board

– alligator clips

IMG_2412

To build a successful “piano” we need to be able to connect the digital pins to ground individually. This will be done by cutting individual “keys” into the paper which will become the triggers or buttons.  First its good to decide on the size and cut the basic shape. I’m going to make a basic rectangle piano with 6 keys.

Basic Design

Cut down to size and figure out how the “buttons” will be made. For this simple example I cut the construction paper to a small rectangle about 6″ x 3″, then I folded it in half the long way so it is now 6″x1.5″.

IMG_2413

Cut the individual keys

IMG_2414

Run the ground “wire”

All the keys need to be able to connect to ground (GND). Since all the keys are in-line I just need to run one long piece of copper tape which will go under all of the keys.

Keep the tape a little longer than it needs to be so it can be folded around the back. This makes for a better surface to connect to with the alligator clips later.

IMG_2416

Now run copper tape for each individual key. Make sure that the tape is long enough to fold around to the back side of the key where it will make contact with the GND copper tape.

IMG_2418

IMG_2419

Connect

Connect the GND to the ground copper tape which runs under all the keys.

Connect the individual keys to the desired digital inputs.

The example patch “simple-synth” makes each digital input control a different pitch to be played.

The fun has just begun!

Try other examples in the sound clippys example folder to explore different sounds and ways of using digital inputs.

Pd Basics

Pd

When you first open Pd you get the “console window”. It looks something like this:

Screen Shot 2015-03-16 at 4.15.22 PM

This is not a Pd program, its just where messages will be printed to. Also, notice the check box in the top right that says DSP. Pd will only produce sound when this box is checked! So go ahead and check it. Also remember you can uncheck it if you need to make the sound stop!

Audio Settings

Its also worth noting Pd will only produce sound if you have working speakers or headphones and Pd is set up to use them. To check the audio settings go to Media > Audio Settings. Here you can make sure Pd is using the input and output device you want to use.

Screen Shot 2015-03-21 at 10.36.18 AM

Create a Patch

To create a new patch select File > New. This will give you a blank window, this is the patch canvas where you do your patching.

Go to the Put menu and add an object. This gives you an empty box which will follow your cursor around. Once you click it is placed and you can type a name or value to define what kind of object you want. This is a good time to try the other options in the Put menu, go ahead…

One thing to know about the Pd and the mm library is that there are help files. To learn about a particular object, right (control) click on an object and select help.

For a more detailed Pd overview I recommend Floss Manuals overview here.

Mapping Inputs to Control Sound

Modular-Muse Interface Objects

The modular-muse library includes a set of interface objects which make it easy to connect to the board and read the inputs into Pd. These objects with work with any Arduino compatible device as long as the Arduino is programmed with the modular-muse firmware.

[mm]

The [mm] object is what communicates between the Sound Clippy (or Arduino) board and Pd. Once connected to the [mm] object the Sound Clippy board sends data over a serial port connection about the state of its inputs.

Screen Shot 2015-07-14 at 2.44.33 PM

Tutorial: Connecting to the Sound Clippy board

[mm-digital]

The [mm-digital] object is for reading the state of the digital pins on the Sound Clippy board. The 12 pins can all be read separately and be used to trigger audio file playback, sound envelopes, enable/disable effects among other things.

To read a digital pin create a new object and type

[mm-digital #]

Screen Shot 2015-07-13 at 11.33.51 PM

where the # is the port number you want to read. The digital pins on the Sound Clippys board are on the side of the board that looks a little like a piano:

clippys-digital-piano

 

the digital inputs are labeled with D2-D13.

The [mm-digital] object has two outputs. The left outlet reperesnts the currents state of the pin and outputs a number, a 0 or a 1 depending on if the port is open or connected to ground in hardware.

This value of 1 or 0 can be used directly to turn on and off objects with an on/off state like the metronome object or the [mm-adsr~] object.

While the right outlet sends a “bang” message when the value changes from 0 to 1. This is good for triggering events

Take a look at the digital input examples for more ideas – or read about creating a paper piano hardware controller.

[mm-analog]

The [mm-analog] object is for reading the value at the analog inputs to the Sound Clippy board.  The 8 analog inputs can all be read in separately and used to control different parameters – or even multiple parameters. For more info about interfacing to the analog inputs in hardware see: Analog Inputs for Expressive Control

Mapping Inputs

With an analog input connected its time to map it to an audio parameter.  From the examples, open the patch “analog-filter” and connect the board. Analog port 0 is configured to control the cutoff frequency of the filter between  200 and 2000 Hz.

Screen Shot 2015-07-14 at 3.49.26 PM

This was created by creating a new object and typing:

[mm-analog 0 200 2000]

Creation Arguments

There are three numbers after the object name [mm-analog]. These are called creation arguments. For this object the first is the port number, next is the low value and finally is the high value. These values can be set accordingly depending on what the sensor is going to be used for. In this case a low-pass filter cutoff frequency is being set between 200 and 2000 Hz.

Sensor Calibration

The analog sensor is automatically calibrated to use the range specified in the [mm-analog] object. The object will keep track of the lowest and highest values seen by the analog port and will map the output to be within the specified range.

To calibrate the sensor – click the “calib” button (or send a bang to the “calibrate” inlet) and exercise the sensors full range. e.g. if its a light sensor, cover it for full darkness and shine a light on it to get the upper end.

Now that you know how to use and map analog inputs to any range – think about the elements of sound again – what do you want to be able to control? Try controlling some different audio effects or sound synthesis parameters!

Connecting to the Sound Clippy Board

[mm]

The [mm] object is what communicates between the Sound Clippy (or Arduino) board and Pd. Once connected to the [mm] object the Sound Clippy board sends data over a serial port connection about the state of its inputs.

 

 

 

 

 

Screen Shot 2015-07-14 at 2.44.33 PM

Connecting

To connect the [mm] object to the Sound Clippy board connect a USB cable between the board and the computer. Open an example such as simple-synth.pd from the sound-clippy folder in the modular-muse library or create a new patch and add the [mm] object to it.

Click on the “list” button. This asks Pd to list all the serial ports it sees. The list is printed out to the Pd window. Here is my Pd window after clicking “list”:

Screen Shot 2015-07-14 at 2.47.00 PM

Note: this will look different on different operating systems (this is from a Mac).

The serial port with the “usbserial” in it is the Arduino. In this case it is on port 4. If you’re not sure which one is the Arduino, try unplugging the USB cable from the device, click list, plug it back in, and click list again. You should see a port which is not in the list when unplugged and is in the list when plugged – this is the one you want!

Back in the [mm] object I change the “port” number box to 4. It should connect to the board with a corresponding “opened serial line device #” message in the Pd window.

You should now be able to use the [mm-digital] and [mm-analog] objects to read the inputs from the board. For more on using these see Mapping Inputs.


 

If you are having trouble communicating with the Sound Clippy board, you may need to make sure the correct firmware is installed. Programming the Sound Clippy Firmware

Programming the Arduino

Programming the Arduino

The modular-muse interface objects can be used to read digital and analog inputs from any Arduino compatible device as long as it is programmed with the modular-muse Arduino firmware.

Once it is programmed once it does not need to be programmed again to continue working with the modular-muse Pd library.

To program the Sound Clippy board with the modular-muse firmware – download and install Arduino.  Open the modular-muse Arduino code in the modular-muse library/arduino-code folder. Select the correct board (Nano) and port (probably says something like usbserial and some other numbers and letters, for Windows it will be a comm port) from the Tools menu.

Click the Upload arrow at the top of the code window.

Musical Gestures and Layout

Musical Gestures

Musicians have control over a lot of different aspects of sound that they use to create expressive, dynamic music.  A violinist alters their bowing speed and pressure to effect timbre, a horn player can create a swell in volume, a vocalist varies their vibrato. This is what makes music interesting, it gives it variation over time. With computer generated music, controlling these parameters is the difference between static, lifeless sounds and interesting expressive musical sounds.

Gestures in Digital Music Instruments

With acoustic instruments the connection between the physical gesture and the resulting sound is obvious – its tied directly to the way the instrument is played. With digital music instruments there is no direct physical mapping – the mappings are up to the designer and is done in software.  However, this doesn’t mean you should make arbitrary mappings! In order to create something that is playable, or even intuitive it’s important to consider how a player wil play a new interface.

Some questions to consider:

– What physical gestures do you want to use to play or interact with your instrument?

– What does each hand do? What about other parts of the body?

– Is the instrument to be held, set on a table, around the neck with a strap or some other interaction? Where do the various inputs (buttons, switches and sensors) go?

– What elements of music or sound parameters do the different interactions control?

– Is there a metaphor for this gesture?

– And finally, for each gesture or sound parameter under control, does it need to be a digital or analog input?

Once you have some ideas to explore – its time to try creating an interface and do the mapping in software.

Go to – Mapping Inputs to Sound – for more on how to use the modular-muse interface objects to bring in data from buttons, switches and sensors.

Analog Inputs for Expressive Control

Analog Inputs for Expressive Control

Unlike digital inputs which have 2 states (i.e. on/off, high/low, 1/0), analog inputs allow for continuous* control over a range of values.  This allows for more subtle control of sound parameters than simply turning a sound on or off.

Elements of Music

Music has a lot of richness and variety for us to enjoy. Much of that has to do with the development of instruments over centuries. Strings, woodwinds, brass and percussion comprise the traditional orchestra families. Each produces sound in unique ways and have different playing techniques which contribute to a wide range of timbres and means of expression.

In addition to the variety of instruments and sounds – compositional aspects have an enormous effect on how we experience music.

What are some elements of music?

Here’s my list: pitch, harmony, scale, melody, rhythm, dynamics, timbre

Others can be defined but for now lets use this list to consider how these elements might map to sound parameters – as we might think about controlling them with computers.

Elements of Music Related Sound Parameters
pitch (notes) frequency (rate)
harmony (more than one note) frequency ratios, or series of intervals played together
scale series of intervals, a collection of notes
melody sequence of pitches
rhythm tempo, patterns, loops
dynamics change in volume, articulation envelopes
timbre waveform, synthesis technique, filter

Much of these are enabled by instruments with a wide range of sounds. These include the more obvious things like: low to high,  soft and LOUD. But also enable playing which emulates more expression, and even emotion. For instance can you imagine music being sweet or angry; smooth or bouncy?

Create expressive controllers with Sound Clippys board

Read hardware inputs and map them to sound using the Interface Objects

Digital Inputs – Making a Paper Piano

To create digital inputs for the Sound Clippy board we need to create an electrical switch which can be turned “on” and “off”. In this case, “on” means connected to GND and “off” means disconnected from GND.

Digital Inputs

Each digital input on the board can be triggered individually. Each port has a number printed on the board indicating which port it is.

The [mm-digital] object uses the number after the object name to select the port. For instance, if you want to read port D2, add a new object and type [mm-digital 2] – you should see this:

Screen Shot 2015-07-13 at 11.33.51 PM

With the Sound Clippy board connected, when a digital input is connected to GND through an alligator clip or with another conductor you should see an “X” in the corresponding [mm-digital #] object toggle box.

Alligator clip switch

Open up simple-synth.pd from the modular-muse/examples/sound-clippys folder and connect to the board. (TODO: how to connect)

– Clip an alligator clip to one of the GND  ports (they’re all the same).

– Touch the other end of the alligator clip to one of the digital pins

digital00

When you touch the alligator clip to the digital pin you’re making an electrical connection between the input pin and ground which the board can detect. This is sent to the corresponding [mm-digital] object.

Paper Piano

You can extend the alligator clip wires by connecting to other conductive materials. Copper tape is a good conductor and is easy to work with.

materials:

– construction paper

– copper tape

– Sound Clippys Board

– alligator clips

IMG_2412

To build a successful “piano” we need to be able to connect the digital pins to ground individually. This will be done by cutting individual “keys” into the paper which will become the triggers or buttons.  First its good to decide on the size and cut the basic shape. I’m going to make a basic rectangle piano with 6 keys.

Basic Design

Cut down to size and figure out how the “buttons” will be made. For this simple example I cut the construction paper to a small rectangle about 6″ x 3″, then I folded it in half the long way so it is now 6″x1.5″.

IMG_2413

Cut the individual keys

IMG_2414

Run the ground “wire”

 

All the keys need to be able to connect to ground (GND). Since all the keys are in-line I just need to run one long piece of copper tape which will go under all of the keys.

Keep the tape a little longer than it needs to be so it can be folded around the back. This makes for a better surface to connect to with the alligator clips later.

IMG_2416

Now run copper tape for each individual key. Make sure that the tape is long enough to fold around to the back side of the key where it will make contact with the GND copper tape.

IMG_2418

IMG_2419

Connect

Connect the GND to the ground copper tape which runs under all the keys.

Connect the individual keys to the desired digital inputs.

The example patch “simple-synth” makes each digital input control a different pitch to be played.

The fun has just begun!

Try other examples in the sound clippys example folder to explore different sounds and ways of using digital inputs.

Tutorial: bi-directional communication between Pd and Arduino (part 3 of 3 – outputs)

This is part 3 of the bi-directional tutorial.

Pd and Arduino code for all 3 parts of this tutorial can be downloaded at github here.

Part 1 covered the basics of sending digital data to Pd. Part 2 adds analog data. This part will discuss ways of sending values from Pd to Arduino.

This covers two possible approaches for sending data from Pd to an Arduino. The first sends a packet of 3 values every time, this can be expanded to include more as needed. The second approach sends a port number followed by a value to set, which also can easily support expanding to as many outputs as you may need.

Approach 1 : sending a packet of 3 values

Code for this approach is found in the sub-folder PdToArduino1.

Approach 1: Pd Patch

This time I will start on the Pd side since thats where data is coming from. The basic idea is to send a series of values, in this case 3 values, on a regular basis. This is done by using a [metro] object which will trigger the values to be sent. A [t b b b] object is used to order them. Pd sends data out from right to left so the first value you want to be sent should be triggered by the far right outlet of the [t b b b] object.

Here is what it looks like:

Screen Shot 2015-02-25 at 4.25.02 PM

The [s toArduino] object has a corresponding [r toArduino] object connected to the [comport] object.

If you needed more outputs you would add another number box and float, and the optional clip object in between as shown above, connected to the [comport]. Then you just need to add a “b” in the [t b b b] object for each additional output.

Approach 1: Arduino code

With this technique the Arduino code will use an array of 3 values to capture these 3 values and then write them to output pins. Note that if you are using digital outputs instead of PWM you can just send 1s and 0s and it will work the same.

With this approach its important that the size of the array for storage and the for loop parameters match the number of bytes sent from Pd. At the top of the Arduino sketch is:

#define NUMBYTES 3

Just set this value to equal the number of bytes being sent by Pd (i.e. the number of [f] objects being sent to the [comport] by the [t b b b…]) object.

The serial parsing looks like this:

Approach 2: Sending a port # and a value

Code for this approach is found in the sub-folder PdToArduino2.

Approach 2: Pd Patch

In this approach, two values are sent each time. First the output number,  followed by the value to send to that output. A couple extra objects, resample and change are used to make sure the serial port doesn’t get overloaded by too much data.

Screen Shot 2015-02-25 at 4.32.27 PM

When the number box on the top left changes it will be sent with a 0 preceeding. The first value (0 in this case) will be parsed by the Arduino to address an array of output pins.

Approach 2: Arduino code

The Arduino code for this approach uses the first value read to the serial buffer to address the outputPins array and the second is the value to set.

Screen Shot 2015-02-25 at 4.34.08 PM

Putting it all together

By combining the code shown in part 1 and part 2 of this tutorial you get bi-directional communication receiving both digital and analog data from Arduino to Pd as well as sending data from Pd to Arduino.

The combination of all of these can be downloaded at github here.

Tutorial: bi-directional communication between Pd and Arduino (part 2 of 3 – adding analog)

This is part 2 of getting bi-directional communication going between Arduino and Pd. This adds a second set of data from Arduino. In this case its reading analog pins. See part 1 for more details on this approach.

Pd and Arduino code for all 3 parts of this tutorial can be downloaded at github here.

To add analog values to the previous example its just a matter of creating another array to store the analog pins, another loop to read them and print them to the serial port. Again, you can just add to the array analogPins[] to expand this to support more (or fewer) pins.

The global variables and setup() function look like this:

Screen Shot 2015-02-25 at 3.53.36 PM

The main loop does the same thing we did with the digital values. Reading each pin and sending a print to the serial port. The packet starts with a capital “A” and ends with a lower-case “a”.

Screen Shot 2015-02-25 at 3.53.47 PM

Pd Patch to parse digital and analog inputs

Now we just need to copy the part of the patch that does the digital parsing and change the creation arguments in the [parse-start-stop] object to be 65 and 97 which are the ASCII code values for “A” and “a”.

You need an “f” in the [unpack] object for every value you are reading. So if you have

Screen Shot 2015-02-25 at 3.58.30 PM

Everything else is the same as in part 1.

Next up

The third part of this tutorial adds data going back to Arduino from Pd.