Logo MIDI Widgets

Spatially anchored controllers


Thanks for taking the time to read through this documentation.

Controllers in MIDI Widgets can be positioned anywhere in your physical spaces and will stay anchored where you put them. MIDI Widgets will recognize different rooms and locations and restore your controller designs exactly as you left them, even across restarts.

MIDI Widgets was designed and created specifically for Apple Vision Pro and will only work on those devices.

MIDI Widgets Tutorial for Subsequent 37

Shared Space Mode

When the MIDI Widgets app is launched, you start in Shared Space Mode and see the main window.

Scenes Panel

The main window contains a list of scenes that group together a collection of controls and their configuration. This allows you to create different layouts for different use cases in the same physical spaces. When MIDI Widgets starts for the first time, an empty Home scene is automatically created for you.

Main Window Scenes

One scene is always selected in the scenes panel and most of the buttons below the list act upon that selected scene. If you delete all the scenes, a new Home scene will automatically be created for you.

The scenes panel provides the following buttons:

To select any of the other panels in the main window, look at the icons on the left of the window and select one. If you don't make a selection, after a brief delay the icons will expand to reveal their labels.

Main Window Sidebar

MIDI Panel

The main window MIDI panel allows you to select which MIDI devices will be used for MIDI output of the selected scene. The MIDI devices that will appear here are the ones that are known to visionOS. At the time of writing, these fall into three categories: virtual app-to-app MIDI, WiFi MIDI network sessions, and Bluetooth LE MIDI.

Main Window MIDI

Virtual MIDI

MIDI Widgets automatically creates its own MIDI Widgets virtual MIDI port through which scenes will always send their MIDI output regardless of the settings in this panel. This can be convenient to select as a MIDI input in other apps without having to perform any configuration changes in MIDI widgets.

NOTE: visionOS 1.x only supports anchoring virtual objects when activating an immersive space, but prevents the windows of other apps to be visible at the same time. In order to control another app on the same device with MIDI widgets, please make sure to enable that app's Background Audio (this terminology varies) option in its settings so that it continues running while MIDI Widgets enters immersive space.

Network MIDI

MIDI Widgets always creates a MIDI Network Session that is ready to be configured on the same local network. The details of how to get the networked MIDI to their intended devices is dependent on how you participate in that networked MIDI session and which devices you use to do so.

There are many tutorials and articles available online that can walk you through the steps, here are two resources to get you started:

Bluetooth LE MIDI

Bluetooth LE MIDI (BLE MIDI) is the easiest way to directly connect your Vision Pro and MIDI Widgets to a hardware device. Your Mac, iPhone and iPad all support BLE MIDI, and any hardware MIDI device can easily become BLE MIDI capable by connecting a CME dongle to it.

The Setup Bluetooth MIDI button allows you to connect to BLE MIDI devices.

A BLE MIDI device can either scan for other devices to connect to or advertise itself as being available for connections. Once a BLE MIDI device connection is made, it's available for all the application on the Vision Pro as new device-wide MIDI ports are created that are available to any app.

NOTE: Creating a BLE MIDI connection doesn't implicitly select that connection's MIDI device as an output for your MIDI Widgets scene. Please make sure to enable the BLE MIDI output in the MIDI panel.

Bluetooth Config

Bluetooth ScanBluetooth Advertise

As with Network MIDI, the details of how to make those connections with the actual devices varies significantly.

Here are some resources to get you started:

Global Settings Panel

The main window Global Settings panel provides configurable options that apply to all scenes.

Main Window Global Settings

The following options are available:

About Panel

The main window About panel provides information about the MIDI Widgets version, as well as handy links.

Main Window About

NOTE: Please consider leaving MIDI Widgets a rating and review in the App Store. It only takes a couple of minutes from your time and makes a significant difference in the success of MIDI Widgets.

The more successful MIDI Widgets is, the more time I can spend on it, and the better it is for you.

Everyone wins!

Immersive Mode

When a MIDI Widgets scene is opened, you enter immersive mode.

If the scene has no controllers, the Create & Position tool will automatically be selected, otherwise the last active tool will remain available. By default, this is the Control & Send MIDI tool.

Tool HUD

In immersive mode, MIDI controllers can be created anywhere in your physical spaces. Therefore, MIDI Widgets is designed so that you always have your tool palette with you. Raising your wrist (left wrist by default, right wrist configurable in global settings) will show a circle with the currently active tool.

MIDI Widgets has four tools available:

Create & PositionEdit & ConfigureControl & Send MIDIRe-position Scene

Tool HUD Create CollapsedTool HUD Edit CollapsedTool HUD Control CollapsedTool HUD Reposition Collapsed

Tool HUD Create ExpandedTool HUD Edit ExpandedTool HUD Control ExpandedTool HUD Reposition Expanded

The bottom left icon leaves immersive space and goes back to shared space, while the bottom right icon collapses the tool HUD again into a compact circle.

Let's dig more into each of the four tools.

Create & Position

When the Create & Position tool is active, you'll see transparent blueprints of the controllers that you can create with MIDI Widgets.

Version 1.0 supports faders, button and knobs, additional types will be added in future updates.

Create Transparent Blueprints

NOTE: if you don't see the transparent blueprints, press and hold the digital crown on your Vision Pro to recenter based on your current physical position.

To create a new instance of a controller, look at the transparent blueprint, pinch and drag away from it. An opaque version of the controller will appear and you can position it anywhere in your space.

You can always re-position a controller by looking at it, pinching and dragging. When pinching with both hands, you're also able to rotate the controller with a horizontal circular motion.

When you move a controller close to another one, it'll automatically align itself and snap to the edges of the other controller. This makes it very easy to build out larger control surfaces with many different controllers.

Create Controller Instance

Edit & Configure

The Edit & Configure tool allows you to change the colors, behavior and layout of a single controller. With the Edit & Configure tool active, look at the controller you want to configure and tap your fingers together. The edit window for that controller will appear above it, oriented towards your viewpoint. To close the edit window, either tap on that controller again or use the close button in the top right corner.

The edit window has three panels: Colors, MIDI and Layout. Display either of these panels by selecting the correspond button in the top bar.

Colors Panel

Five parts of a controller are available for color configuration: the horizontal base, the configurable name, the markers on the base, the control itself and the accessory shape that's connected to some controls.

Colors Base ColorColors Name ColorColors Control Color

Based on your preference, you can switch between HSB and RGB color selection. Each color also has a copy and a paste button next to it, making it very easy to propagate similar colors.

At the bottom of the color panel, copy and paste buttons allow you to transfer the entire color scheme to other controllers. When you create a new controller with the Create & Position tool, it will use the latest color scheme that was copied or the default color scheme if none is available in the clipboard.

The reset button in the bottom right corner, resets the colors of the current controller to the default color scheme.

MIDI Panel

Four types of MIDI messages can be sent by a MIDI Widget controller: Control Change, Channel Pressure, Program Change and Pitch Bend. Each of these message types have slightly different configuration options.

MIDI Control ChangeMIDI Channel Pressure

MIDI Program ChangeMIDI Pitch Bend

NOTE: If you're not familiar with these MIDI message types, I strongly recommend you familiarize yourself with these fundamental building blocks of MIDI. The MIDI Association's website is a good starting point.

Layout Panel

The layout panel offers additional layout features to a controller.

The bottom left corner has a duplicate button that creates a new instance of the selected controller after which it's place to the right side.

The bottom right corner has a delete button that will completely delete the selected controller after confirmation.

Layout Panel 

Control & Send MIDI

This is the tool you'll likely use the most. It is also the tool MIDI Widgets defaults to when a scene is loaded with existing controllers.

When in Control & Send MIDI tool, no positioning, configuration or layout changes are possible to any of the controls.

Simply look at a control, pinch and drag to change the faders or knobs, or pinch and tap to change the buttons. When dragging, you have the option to either drag in the natural direction of the control, as it's placed in your physical space, or to vertically drag upwards or downwards. Which direction is used depends on the initial drag motion you make during one particular gesture.

MIDI Widgets supports multiple concurrent gestures. Feel free to drag multiple faders with both hands, to hold a knob while you trigger a button, or any combination you can think of. Note that visionOS only processes gesture updates when your hands can be seen by the device. It's sometimes easy to get forget where your hand is while you're looking for other controllers to interact with.

Re-position Scene

Sometimes you want to move all the controllers in your scene to a different position or to rotate them. The Re-position Scene tool allows you to do that without have to move each controller individually.

To move the position of all controllers, simply look at one of the controllers, then pinch and drag to the new position.

To change the orientation of your scene, look at the controller that should be the pivot point, then pinch with both hands and perform a horizontal circular motion.

Thanks for reading this manual!

MIDI Widgets is at the forefront of technology, exploring a new paradigm that has never been explored before.

Some things might not work as you expect, might not be stable, or even be unreliable. This can be due to MIDI Widgets, but also due to the Vision Pro or visionOS, or any other device or application that you're interacting with.

MIDI Widgets is made by me, a single passionate developer, in my spare time. MIDI Widgets will likely never sell enough to recoup the amount of time and resources I've spent on it. Please take that into account before you contact me for support requests. There are many other resources and communities available online.

That being said, if you think there's a problem with MIDI Widgets, please contact me before leaving a bad review on the App Store. I'll try my best to help you within the time that I have available.