Virtual MIDIbox SEQ V4 for iPad

[Narwhal]

If ipMIDI becomes a defacto standard, it shouldn't be a big problem to provide it as an alternative option. The protocol is simple but primitive compared to OSC. I guess that the transfer performance is almost identical.

This is a controller which uses the Bonjour based protocol: http://futuremusic.c...or-apples-ipad/

According to the german introduction, no other iPhone/iPad application uses it yet, but the big advantage should be Plug&Play (no driver installation required)

Interesting.. if you turn up and info on how that's accomplished I certainly would like to take a peek at it. Maybe something in gamekit.. I'll snoop around there.

[Narwhal]

** Like and idiot I edited, and amazingly deleted, my original message here rather than replying. So here is the new complete message with all the original info:

Here we go:

The midi service is browsable with bonjour on the iPhone/iPad. I just did it using a sample app called BonjourWeb that was installed from the xcode documentation.

In the file "BonjourWebAppDelegate.m" I changed:

#define kWebServiceType "_http._tcp"[/size]

[size=2]to[/size]

[size=2]#define kWebServiceType "_apple-midi._udp"

Then lo and behold, when I ran it my mac showed up in the browser.

If I click on the computer name it attempts to open the service on the phone, but it has no idea how to deal with it.

Here are some links to details for the apple-midi protocol:

CoreAudio mailing list 2008

RTP MIDI Page

RFC 4695

RFC 4696

Sound on Sound article about how to setup the standard Apple network port driver

MIDIShare Project that includes RTP Support

RTP-MIDI seems to be far more advanced than ipMIDI as it is capable of handling lost packets.

My current belief is that the MIDIpad guys have used this code to engineer their own connection to the apple-midi protocol. RTP does show up a few times in the dev docs, so maybe there is more there. The hunt continues...

I just found more of a clue that the solution that MIDIpad is using is a custom one. One of MIDIpad's authors is Tobias Erichsen who has also contributed to the wireshark plugin that decodes RTP-MIDI and AppleMIDI session protocols. The source to these plugins is definitely illuminating as to the session protocol, and he has kindly included a capture file in the wireshark bug report.

Edited April 22, 2010 by Narwhal

[napierzaza]

Whoa, this is nice to see!

I'm a amateur iPhone/iPad developer and I'm really excited to see some really professional music-related work going up. My next project plan was to build some kind of OSC music player. But I still can't even figure out how to get an OSC running in it at all. Ah well, one day. Everyone of my my apps has had something that was really difficult to get over.

I'm definitely loading this on my iPad when I get home.

Do you plan on working on the graphics a bit more?

Edited April 23, 2010 by napierzaza

[chrisd86]

hi guys, have been developing an controller app of my own, just out of interest, can anyone think why this zero-conf method midimap is using has never been used before?! seems like the most obvious solution, but due to never hearing about it, i assumed it wasnt possible!

[Narwhal]

hi guys, have been developing an controller app of my own, just out of interest, can anyone think why this zero-conf method midimap is using has never been used before?! seems like the most obvious solution, but due to never hearing about it, i assumed it wasnt possible!

Simply because the Apple-midi session protocol is not public. That means that there is no public specification for how to connect to a Mac that is running that network midi driver. As I show above, the only reason the MidiPad app is trying to do it is because one of the authors of the app has been involved with one of the protocols used (he is thanked in the RTP-MIDI document), and is also known to be involved in reverse engineering the Apple-MIDI session protocol (See the WireShark bug database links). So as of right now he is probably one of very few people in the world, outside of Apple, that knows how connect to the Mac's network midi driver.

[TK.]

Today I sent the first OSC based MIDI messages from my iPad :)

Macintosh:osc_midi_proxy TK$ ./osc_midi_proxy 192.168.1.108 10000 10001

Connecting to 192.168.1.108 on port 10000

Receiving on port 10001

in stat: : No such file or directory

in stat: : No such file or directory

[ 0] CoreMIDI, IAC Driver IAC-Bus 1 (default input)

[ 1] CoreMIDI, IAC Driver IAC-Bus 2 (input)

[ 2] CoreMIDI, Ploytec GM5 www.midibox.org Anschluss 1 (input)

[ 3] CoreMIDI, Ploytec GM5 www.midibox.org Anschluss 2 (input)

[ 4] CoreMIDI, Ploytec GM5 www.midibox.org Anschluss 3 (input)

[ 5] CoreMIDI, Ploytec GM5 www.midibox.org Anschluss 4 (input)

[ 6] CoreMIDI, Ploytec GM5 www.midibox.org Anschluss 5 (input)

[ 7] CoreMIDI, Ploytec USB MIDI Junction II Anschluss 1 (input)

[ 8] CoreMIDI, Ploytec USB MIDI Junction II Anschluss 2 (input)

[ 9] CoreMIDI, Ploytec USB MIDI Junction II Anschluss 3 (input)

[10] CoreMIDI, Ploytec USB MIDI Junction II Anschluss 4 (input)

[11] CoreMIDI, Ploytec USB MIDI Junction II Anschluss 5 (input)

Type input number: 0

[12] CoreMIDI: IAC Driver IAC-Bus 1 (default output)

[13] CoreMIDI: IAC Driver IAC-Bus 2 (output)

[14] CoreMIDI: Ploytec GM5 www.midibox.org Anschluss 1 (output)

[15] CoreMIDI: Ploytec GM5 www.midibox.org Anschluss 2 (output)

[16] CoreMIDI: Ploytec GM5 www.midibox.org Anschluss 3 (output)

[17] CoreMIDI: Ploytec GM5 www.midibox.org Anschluss 4 (output)

[18] CoreMIDI: Ploytec GM5 www.midibox.org Anschluss 5 (output)

[19] CoreMIDI: Ploytec USB MIDI Junction II Anschluss 1 (output)

[20] CoreMIDI: Ploytec USB MIDI Junction II Anschluss 2 (output)

[21] CoreMIDI: Ploytec USB MIDI Junction II Anschluss 3 (output)

[22] CoreMIDI: Ploytec USB MIDI Junction II Anschluss 4 (output)

[23] CoreMIDI: Ploytec USB MIDI Junction II Anschluss 5 (output)

Type output number: 13

HINT: next time you could select the MIDI In/Out port from command line with: ./osc_midi_proxy 192.168.1.108 10000 10001 --in 0 --out 13

MIDI IN 'CoreMIDI: IAC Driver IAC-Bus 1' opened.

MIDI OUT 'CoreMIDI: IAC Driver IAC-Bus 2' opened.

Proxy is running!

### Received 16 bytes

[/midi1] timetag 0.1 (1 args), Method Arg: 0x00000000

[/midi1] 0: 643C9009 (MIDI)

[MIDI_OUT] 90 3C 64

### Received 16 bytes

[/midi1] timetag 0.1 (1 args), Method Arg: 0x00000000

[/midi1] 0: 003C9009 (MIDI)

[MIDI_OUT] 90 3C 00

### Received 16 bytes

[/midi1] timetag 0.1 (1 args), Method Arg: 0x00000000

[/midi1] 0: 643C9009 (MIDI)

[MIDI_OUT] 90 3C 64

### Received 16 bytes

[/midi1] timetag 0.1 (1 args), Method Arg: 0x00000000

[/midi1] 0: 003C9009 (MIDI)

[MIDI_OUT] 90 3C 00

### Received 16 bytes

[/midi1] timetag 0.1 (1 args), Method Arg: 0x00000000

[/midi1] 0: 643C9009 (MIDI)

[MIDI_OUT] 90 3C 64

### Received 16 bytes

[/midi1] timetag 0.1 (1 args), Method Arg: 0x00000000

[/midi1] 0: 003C9009 (MIDI)

[MIDI_OUT] 90 3C 00

### Received 16 bytes

[/midi1] timetag 0.1 (1 args), Method Arg: 0x00000000

[/midi1] 0: 643C9009 (MIDI)

[MIDI_OUT] 90 3C 64

...

[/code]


If you want to try this out:

- update the repository

- adapt the hard-coded IP address in Classes/OscServer_Wrapper.m (IP address of the host, will be configurable later)

- compile and upload the application to your iPad

- compile the OSC<->MIDI proxy under tools/osc_midi_proxy with "make -f Makefile.osx"

- start it with "./osc_midi_proxy <ip-address-of-your-ipad> 10001" on your Mac and route OSC packets to a MIDI bus (or external MIDI interface)


I haven't checked the performance (latency/jitter) yet, but will continue with this next week (after holidays...)


Best Regards, Thorsten.

[DrBunsen]

Fabulous work TK!

I have one question:

I have recently been looking at hardware MIDI out from the iPad/iPhone dock connector. In searching, I have found one commercial product, the MIDI Mobilizer from Line 6, one Apple-approved reference design hardware RS-232 port, and some DIY schematics for serial I/O.

Is there any prospect of your app supporting hardware MIDI I/O direct from the iPad?

[to-pse]

Hi everyone,

as I have recently discovered your discussion concerning network-MIDI,

I just wanted to give you guys a short hint, that I have just uploaded the

first public release of my network MIDI driver.

This driver is meant for Windows XP up to Windows 7, supports 32bit

and 64bit and is compatible to the network MIDI driver included in

every OS x since "Tiger".

If you are interested to give it a try - just take short look at my new website:

www.tobias-erichsen.de

Best regards,

Tobias

[TK.]

Hi Tobias,

thank you for releasing the windows driver! :)

I would like to test it, but don't know how to use the RTP-MIDI protocol on iPad.

Are there undocumented functions available in iOS, or did you implement the protocol stack for iOS as well?

Best Regards, Thorsten.

[to-pse]

Hi Tobias,

thank you for releasing the windows driver! :)

I would like to test it, but don't know how to use the RTP-MIDI protocol on iPad.

Are there undocumented functions available in iOS, or did you implement the protocol stack for iOS as well?

Best Regards, Thorsten.

Hi Thorsten,

unfortunately Apple did not implement the stack within iOS. So I had to reimplement it in Objective C...

Best regard,

Tobias

[marcora]

Where can I download SEQ V4 for the iPad?

[sineSurfer]

hey, good news!, looks like iOS 4.2 will have coreMidi support

marcora:,over here: http://svnmios.midibox.org/listing.php?repname=svn.mios32

[gjvti]

Hi there.

Just FYI

Yesterday I found this iConnect project

which obviously will be available next year and just though that probably it could

find some use with midibox sequencer as well. Probably it could be possible to make

virtual BLM for iPad/multitouch enabled PC to control Midibox hardware in a way Thorsten

did with

I think it could open may possibilities

for different ways to connect music gear.

gj

[TK.]

I've a similar project, but it's a little bit more advanced, because it supports:

• USB MIDI
  
• UART MIDI (up to 3 IN ports and 3 OUT ports)
  
• OSC (requires MBHP_ETH module)
  
• CV (requires MBHP_AOUT* module)
  

Probably it could be possible to make virtual BLM for iPad/multitouch enabled PC to control Midibox hardware in a way Thorsten did with hardware matrix.

Thats already possible by using OSC

My plan is to provide such an app, but there are other higher-prio things in the queue... ;)

We could speed up this if somebody would help me with the graphics!

Btw.: OSC seems to be popular as well, e.g. Pianist Pro supports this:

http://midibox.org/forums/index.php?app=gallery&module=images&section=viewimage&img=474

I'm able to directly control my MBSEQ with this app (w/o using a PC), and it's also possible to route key/pitch/CCs via the OSC port of MBSEQ to a physical MIDI IN/OUT by using MBSEQ

Since this is working w/o problems, and since I'm already able to send/receive OSC messages on an iPad with my own applications, this is probably the route to go.

Best Regards, Thorsten.

[gjvti]

Hi Thorsten!

First I'd like to express my adoration to your

solid sequencer aluminium case - looks really great.

I didn't notice that before :)

I've a similar project, but it's a little bit more advanced, because it supports:

• USB MIDI
  
• UART MIDI (up to 3 IN ports and 3 OUT ports)
  
• OSC (requires MBHP_ETH module)
  
• CV (requires MBHP_AOUT* module)
  

Thats already possible by using OSC

My plan is to provide such an app, but there are other higher-prio things in the queue... ;)

We could speed up this if somebody would help me with the graphics!

Btw.: OSC seems to be popular as well, e.g. Pianist Pro supports this:

http://midibox.org/forums/index.php?app=gallery&module=images&section=viewimage&img=474

I'm able to directly control my MBSEQ with this app (w/o using a PC), and it's also possible to route key/pitch/CCs via the OSC port of MBSEQ to a physical MIDI IN/OUT by using MBSEQ

Since this is working w/o problems, and since I'm already able to send/receive OSC messages on an iPad with my own applications, this is probably the route to go.

Best Regards, Thorsten.

iConnect box, as they say, directly connects to iPhone/iPad

by wire, which also serves as a charger, but yes - it doesn't

support OSC and can't go wireless as I understand. Since I'm

just learning to work with my sequencer, when I have free time,

I haven't dwelled deep into this OSC connectivity - this

is really new for me, but possibility to build

own visual control panel for iPad or multitouch

enabled PC to have full control over MBox sequencer

and other midi gear looks very tempting and promising.

I can't 100% promise, but I think I can give a try to

help you with graphics if you can elaborate this.

Just send me an email or pm what format graphics

you need and what elements should be laid on there.

Best regards

Gunars

[TK.]

iConnect box, as they say, directly connects to iPhone/iPad

by wire, which also serves as a charger, but yes - it doesn't

support OSC and can't go wireless as I understand.

IMHO connecting MIDI via a wire doesn't make on an iPad - computers have been invented for such a usecase, but not the iPad ;)

Since I'm

just learning to work with my sequencer, when I have free time,

I haven't dwelled deep into this OSC connectivity - this

is really new for me, but possibility to build

own visual control panel for iPad or multitouch

enabled PC to have full control over MBox sequencer

and other midi gear looks very tempting and promising.

You won't find so many informations about using OSC for MIDI, thats something that I've developed.

I'm using the same library for MIOS32, Juce (-> MacOS, Windows, Linux), native MacOs and iPad - the code is optimized for embedded systems (and therefore runs much faster than common OSC libraries) and code written for this library is compatible with any of these operating systems. :)

I can't 100% promise, but I think I can give a try to

help you with graphics if you can elaborate this.

Just send me an email or pm what format graphics

you need and what elements should be laid on there.

The problem is, that a complete new user interface has to be developed which is optimized for touch screen entry, because the "emulated frontpanel" isn't really ergonomic.

I've no clear ideas about the UI part yet - it will need some months to work out such a plan, and here the process could be speed up if somebody experienced in creating user interfaces could help.

Best Regards, Thorsten.

[TK.]

Good news: the Juce based virtual BLM can be compiled for iOS

It's properly running on my iPad - next step is to add the OSC code :)

Imperfection: Juce isn't able (yet) to rotate the screen - seems that this has to be hardcoded.

Best Regards, Thorsten.

[TK.]

Update: communication via OSC is working, but graphic output is too slow. :-/

Time to optimize the "LED" handling.

Best Regards, Thorsten.

[TK.]

Last update: I'm now using my own button elements, and the performance is much better.

But it still doesn't run smooth on the iPad; it runs perfectly on the simulator (immediate update of the display).

Since iPad and the Simulator (running on a MBP) communicate with MBSEQ via WiFi, I know that this is neither a MBSEQ issue, nor a protocol issue - it's either related to several Juce layers, or to the slow iPad CPU, or to the slow network interface of the iPad.

For comparison:

pinging an iPad from a MBP (MBP->WiFi->Router->WiFi->iPad):

Macintosh:Desktop TK$ ping 192.168.1.110

PING 192.168.1.110 (192.168.1.110): 56 data bytes

64 bytes from 192.168.1.110: icmp_seq=0 ttl=64 time=26.624 ms

64 bytes from 192.168.1.110: icmp_seq=1 ttl=64 time=48.519 ms

64 bytes from 192.168.1.110: icmp_seq=2 ttl=64 time=72.720 ms

64 bytes from 192.168.1.110: icmp_seq=3 ttl=64 time=100.601 ms

64 bytes from 192.168.1.110: icmp_seq=4 ttl=64 time=143.209 ms

64 bytes from 192.168.1.110: icmp_seq=5 ttl=64 time=42.671 ms

64 bytes from 192.168.1.110: icmp_seq=6 ttl=64 time=84.293 ms

64 bytes from 192.168.1.110: icmp_seq=7 ttl=64 time=92.300 ms

64 bytes from 192.168.1.110: icmp_seq=8 ttl=64 time=112.074 ms

64 bytes from 192.168.1.110: icmp_seq=9 ttl=64 time=38.549 ms

64 bytes from 192.168.1.110: icmp_seq=10 ttl=64 time=63.361 ms

64 bytes from 192.168.1.110: icmp_seq=11 ttl=64 time=81.214 ms

64 bytes from 192.168.1.110: icmp_seq=12 ttl=64 time=105.147 ms

^C

--- 192.168.1.110 ping statistics ---

13 packets transmitted, 13 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 26.624/77.791/143.209/32.190 ms

[/code]






pinging MBSEQ from a MBP (MBP->WiFi->Router->MBSEQ):

[code]

Macintosh:Desktop TK$ ping 192.168.1.112

PING 192.168.1.112 (192.168.1.112): 56 data bytes

64 bytes from 192.168.1.112: icmp_seq=0 ttl=64 time=6.479 ms

64 bytes from 192.168.1.112: icmp_seq=1 ttl=64 time=4.614 ms

64 bytes from 192.168.1.112: icmp_seq=2 ttl=64 time=4.037 ms

64 bytes from 192.168.1.112: icmp_seq=3 ttl=64 time=4.359 ms

64 bytes from 192.168.1.112: icmp_seq=4 ttl=64 time=4.237 ms

64 bytes from 192.168.1.112: icmp_seq=5 ttl=64 time=3.504 ms

64 bytes from 192.168.1.112: icmp_seq=6 ttl=64 time=3.329 ms

64 bytes from 192.168.1.112: icmp_seq=7 ttl=64 time=4.056 ms

64 bytes from 192.168.1.112: icmp_seq=8 ttl=64 time=6.043 ms

64 bytes from 192.168.1.112: icmp_seq=9 ttl=64 time=5.708 ms

64 bytes from 192.168.1.112: icmp_seq=10 ttl=64 time=3.650 ms

64 bytes from 192.168.1.112: icmp_seq=11 ttl=64 time=3.435 ms

64 bytes from 192.168.1.112: icmp_seq=12 ttl=64 time=4.524 ms

64 bytes from 192.168.1.112: icmp_seq=13 ttl=64 time=4.664 ms

64 bytes from 192.168.1.112: icmp_seq=14 ttl=64 time=5.734 ms

64 bytes from 192.168.1.112: icmp_seq=15 ttl=64 time=2.350 ms

^C

--- 192.168.1.112 ping statistics ---

16 packets transmitted, 16 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 2.350/4.420/6.479/1.077 ms

pinging MBSEQ which is directly connected to the ethernet port of a MBP (MBP->MBSEQ):

Macintosh:Desktop TK$ ping 192.168.2.6

PING 192.168.2.6 (192.168.2.6): 56 data bytes

64 bytes from 192.168.2.6: icmp_seq=0 ttl=64 time=1.669 ms

64 bytes from 192.168.2.6: icmp_seq=1 ttl=64 time=1.506 ms

64 bytes from 192.168.2.6: icmp_seq=2 ttl=64 time=1.430 ms

64 bytes from 192.168.2.6: icmp_seq=3 ttl=64 time=1.177 ms

64 bytes from 192.168.2.6: icmp_seq=4 ttl=64 time=1.015 ms

64 bytes from 192.168.2.6: icmp_seq=5 ttl=64 time=1.010 ms

64 bytes from 192.168.2.6: icmp_seq=6 ttl=64 time=0.880 ms

64 bytes from 192.168.2.6: icmp_seq=7 ttl=64 time=0.706 ms

64 bytes from 192.168.2.6: icmp_seq=8 ttl=64 time=1.573 ms

64 bytes from 192.168.2.6: icmp_seq=9 ttl=64 time=1.396 ms

64 bytes from 192.168.2.6: icmp_seq=10 ttl=64 time=1.280 ms

64 bytes from 192.168.2.6: icmp_seq=11 ttl=64 time=1.181 ms

64 bytes from 192.168.2.6: icmp_seq=12 ttl=64 time=1.036 ms

64 bytes from 192.168.2.6: icmp_seq=13 ttl=64 time=0.914 ms

64 bytes from 192.168.2.6: icmp_seq=14 ttl=64 time=0.764 ms

64 bytes from 192.168.2.6: icmp_seq=15 ttl=64 time=1.634 ms

64 bytes from 192.168.2.6: icmp_seq=16 ttl=64 time=1.496 ms

64 bytes from 192.168.2.6: icmp_seq=17 ttl=64 time=1.286 ms

^C

--- 192.168.2.6 ping statistics ---

18 packets transmitted, 18 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 0.706/1.220/1.669/0.293 ms

[/code]


Best Regards, Thorsten.

[gjvti]

Hi Thorsten!

Your iPad virtual BLM looks very

intriguing and interesting.

Sad that it has the issues

with responsiveness.

I'm curious - has someone tested

virtual BLM on multitouch enabled

PC/tablet PC?

gj

[monokinetic]

Damn, it is a shame that you are having problems with the iOS version, this would be a killer app for me.

I have tried pinging my iPad and I get similar average times to you. This seems strange though, I have been using OSCemote and TouchOSC to send OSC to trigger Kyma and I have not had any noticeable datatransfer problems, even when sending a large amount of data (ten fingers with X,Y position and other TUIO stuff) I haven't noticed any timing issues. I realise this is just sending data one way, but a quick Google shows iPads doing similar sequencer editors for Ableton and SuperCollider. Also I could mention that my initial problems with iPad and OSC were solved by updating my Asus wirless router to the latest firmware....

Just brainstorming here, but would it be worth trying a purely OSC based solution to see if the iPad is up to the job? I could quite easily create a TouchOSC layout with the required buttons, that might at least identify if it is a problem with the iPad hardware / Juce software.

Other than that I'm not sure what else could be worth trying....

David

[TK.]

Hi Thorsten!

I'm curious - has someone tested

virtual BLM on multitouch enabled

PC/tablet PC?

Btw.: I don't expect problems with a tablet PC :)

If somebody is interested, I could release the OSC option for the virtual BLM for Mac/Windows/LInux as well

Just brainstorming here, but would it be worth trying a purely OSC based solution to see if the iPad is up to the job? I could quite easily create a TouchOSC layout with the required buttons, that might at least identify if it is a problem with the iPad hardware / Juce software.

Other than that I'm not sure what else could be worth trying....

Such a comparison definitely makes sense.

I've the TouchOSC app already installed, could you please create the appr. layout?

Feel free to define the OSC messages as you want, I'm very flexible.

Best Regards, Thorsten.

[sineSurfer]

Just did this touchOSC layout, added red/green leds on the first row of switches to check if that's the way to go... you need to use OSCulator or something to make this useful? I would do it but can't sync the ipad on the office network.

hehee, now with the file.

virtualBLM.touchosc.zip

Edited September 30, 2010 by sineSurfer

[monokinetic]

Just did this touchOSC layout, added red/green leds on the first row of switches to check if that's the way to go...

Wow that was quick!

you need to use OSCulator or something to make this useful?

Well my understanding was that we could use this to test OSC going from the MBSeq to iPad. I don't see how OSCulator would be involved here?

The problem for me is that I still have not finished soldering my Ethernet for the MbSeq so I can't test it on the hardware. Hopefully during the week I can get it done.

Anyone else got an iPad + Ethernet enabled MbSeq apart from TK? :)

David

[sineSurfer]

Update on the iPad + MIDI front:

Last weekend I tested the iPad with iOS 4.2 + iPad Camera Connection Kit+ midi monitor app + gm5x5x5, and...

IT WORKED!!! , ok... just tested moving things on the app, the gm5x5x5 leds shine away like a christmas tree, I'll have to test midi input too later.

(tried also with a m-audio midisport 4x4, no go)

But somehow the last time I tried to compile mbSEQ I couldn't (like a month ago) so I don't know if it will work without any further modifications, have you tested this TK?