The I-CubeX line of products aims to facilitate the design and implementation of environments and objects that respond to human actions such as walking, reaching, touching etc. as well as environmental parameters such as illumination, temperature, humidity etc.. The response can be in the form of sound, music, video, graphics, animation, robotic movement, etc.. Such responses are generated by I-CubeX actuators and/or 3rd party equipment. While it is a widely used tool for prototyping and experimentation wrt. human interfacing it is also perfectly suitable as a scientific research tool to gather data about movement and behaviour.

Application scope

The I-CubeX line of products only includes tools to capture the human actions and/or environmental variables, the tools to make these signals available to other equipment such as a computer or a musical instrument and tools to create responses using actuators. To communicate with such equipment the MIDI (Musical Instrument Digital Interface) data format is used, transmitted via MIDI cable, USB cable or wireless Bluetooth. Communication via WiFi is implemented using the OSC (Open Sound Control) and WS (Web Socket) protocol. MIDI and OSC are widely implemented protocols for communications between musical equipment which have also found their way into video, graphics and animation software as well as various equipments, such as lights, used in theme parks and live shows. Even though I-CubeX uses the MIDI and OSC data formats, tools are provided to output to devices that use other protocols, enlarging its application scope to gaming, engineering and science.

Components

To accommodate a wide variety of designs of such responsive environments and objects, the I-CubeX environment consists of the following modular components that can be configured in a multitude of ways:

• One or more sensors.
• One or more actuators.
• One or more digitizers.
• Configuration software.

Sensors and Actuators

Sensors and Actuators connect to a digitizer through a miniature, flexible cable with a 3- or 4-pin plug. Sensors can be positioned on the human body or on animals, on walls or floors or on sculptures. The multitude of sensors available from I-CubeX cover almost any human, animal or object motion, as well as a variety of environmental parameters. I-CubeX sensors output either analog voltages or digital I²C signals. I²C is a commonly used bi-directional protocol in robotics. I-CubeX Actuators are controlled via binary logic voltages, PWM (pulse width modulation) signals or digital I²C signals.

Digitizers

The digitizer converts the signal from the sensor to a digital message. A digitizer can convert up to 8 (USB-microDig, Wi-microDig, WiDig) or up to 32 (Digitizer) sensor signals. The Digitizer connects through a MIDI cable to a computer that is equipped with a MIDI interface (see figure 1) or directly to a MIDI capable device (see figure 2). The USB-microDig connects through a USB cable to a computer that is equipped with a USB port (see figure 3). The Wi-microDig connects wirelessly through Bluetooth Classic to a computer equipped with Bluetooth Classic (see figure 5). The WiDig connects wirelessly through BLE (Bluetooth Low Energy) or WiFi to a mobile device or computer equipped with BLE or WiFi (see figure 5 and 6) as well as through a USB cable to a computer that is equipped with a USB port (see figure 3). All digitizers use the MIDI data format to encode the sensor signals, but note that on a computer the USB-microDig, Wi-microDig and WiDig (when connected via USB) make this MIDI encoded sensor data available through a virtual serial port whereas the Digitizer makes the sensor data available through a MIDI port. Digitizers can operate in two modes: host mode, when the digitizer outputs raw, unprocessed sensor data as MIDI System Exclusive messages, and standalone mode, when the digitizer outputs processed sensor data as MIDi Channel Voice messages, using a variety of algorithms. These modes and other features are realized with the firmware, that functions as a kind of operating system and which runs on the digitizer. The WiDig also runs WiConnect and, when operating via WiFi, can output OSC and web socket (WS) messages to transmit sensor data while it can also receive OSC and WS messages to control actuators.

Figure 1. Digitizer connected to computer via MIDI-USB interface.	Figure 2. Digitizer connected to MIDI capable device.
Figure 3. Digitizer connected to computer directly via USB.	Figure 4. Digitizer connected to MIDI capable device.
Figure 5. Digitizer connected to computer via Bluetooth.	Figure 6. Digitizer connected to mobile device via WiFi or BLE (Bluetooth Low Energy).

Software

Various software tools are available to work with I-CubeX (see figure 7). To configure the digitizer for standalone mode run the I-CubeX EditorX software. To configure the digitizer for host mode run the I-CubeX Link software. EditorX and Link communicate with the digitizer via a MIDI port. If the digitizer communicates via a USB or Bluetooth Classic serial port, the I-CubeX Connect software is used to represent the digitizer on the computer as a MIDI port. Connect also functions as a bridge using OSC (Open Sound Protocol, a common IP-based communication protocol) to other applications, possibly running on other computers, or Webhooks for IoT (Internet of Things) applications via the internet. When communicating via BLE (Bluetooth Low Energy) with a digitizer, the BLE-MIDI protocol is used (see figure 8). EditorX sets a variety of parameters in the digitizer such as the type of processing of the sensor signal and which MIDI message to map to. It provides a way to verify if the sensor captures what is required to generate an appropriate response. Once the user has configured the digitizer with EditorX as needed, the MIDI messages can be sent directly to MIDI-capable software, such as GarageBand, Ableton Live or TouchDesigner, or a hardware device to generate a response. In the standalone mode of operation it is then not necessary to use a computer program to process or analyze the sensor data. Using EditorX or Link the sensor data can also be saved on disk for later analysis using 3rd party applications. Our Keys software enables the user to convert the sensor data intelligently to keystrokes, which then in turn can control other software programs such as media players. Link provides a convenient way to configure the digitizer for host mode, and process, analyze, display, record and map the raw sensor data to create more complex output. Plugins for 3rd party software and tools to integrate I-CubeX with a number of programming environments, such as Max, or languages such as C/C++, are available as well. Figure 8 illustrates communication via WiFi with the WiDig using OSC, HTTP and web socket (WS) protocols. Configuration of the WiDig takes place using a web browser.

Figure 7. Typical software, protocols and data routing methods when communicating with a digitizer via a USB or Bluetooth Classic serial port.	Figure 8. Typical software, protocols and data routing methods when communicating with a digitizer via BLE (Bluetooth Low Energy).
Figure 9. Typical software, protocols and data routing methods when communicating with a WiDig via WiFi.

What you need

In summary, to get started with creating a responsive environment or object you will need:

• I-CubeX environment (sensors, digitizer(s), editor)
• MIDI interface (Digitizer), USB port (USB-microDig) and/or Bluetooth interface (Wi-microDig, WiDig)
• Computer(s) (Windows or MacOS)
• MIDI device(s) or software(s) to generate required response(s) (3rd party)

In the help section on our website you can find more gear from a number of 3rd party manufacturers to help you generate a suitable response.

Further reading

There is more to know about I-CubeX, how it has been used, sample applications, more technical details, ..

I-CubeX arose out of a research project in 1995 directed by Axel Mulder, while he was doing his PhD on virtual musical instruments at the Department of Kinesiology, Simon Fraser University, to address the need for better tools for artists to create interactive art and for musicians to more easily create or modify musical instruments. The project led to a paper entitled The I-Cube System: moving towards sensor technology for artists, after which it was commercialized.