Dynamic Stop Operation Problem

[robinfawell]

After nearly 3 years I have a working organ based on a US Conn Church organ and the Italian SCPOP organ design.  The electronics are based on 2 core modules, 5 Din modules and 2 Dout modules.  The modules are programmed in C.

The system sends Midi Note On/Off from the 2 keyboards and pedalboard to a Roland Sound Module (SC 8850).  The Roland unit also receives sysex messages generated by the console "stops" as well as various Midi CC messages.

Everything works as planned except for one very essential feature.

There is a significant delay when the organ stop is changed.  This means that there is a period when no further playing can occur.  There is either a silence  or a hung note during the transfer period of about 2-3 seconds.  This is unsatisfactory.

Most of the sysex messages are about 1000 bytes +- 250 bytes.  There are 2 at 2756 bytes.

I need to solve this problem to make the organ a practical proposition. Here are some possibilities.  I would need an improvement of at least 10 times the transfer speed.

The Roland Sound Module has a USB input as well as Midi.  I could change to a USB or IIC module.

Perhaps the transfer rate of the current system could be speeded up. Is this a practical course of action?  Change to MIOS_SRIO_UpdateFrgSet ?

I would welcome any ideas.

Regards Robin

[TK.]

Hi Robin,

is the debouncing activated in your application? This could explain everything.

There are two solutions: either set it to 0 (in this case, the buttons are not debounced anymore, but the outputs are also not delayed anymore) - or update to MIOS V1.9c

In your case, you have to select the pic18f452/midi/update_with_old_mios.hex file  (don't upload any other of the update package!)

Best Regards, Thorsten.

[TK.]

It just came in my mind, that I misread your posting (MIOS_SRIO_UpdateFrgSet was missleading - it doesn't help).

I think the general problem is the slow transfer speed of MIDI.

Each MIDI byte takes 320 uS, 1000 bytes are taking 0.32 mS, and if your program sends multiple dumps with one button push, we have the explanation, why it takes so long.

Only the number of bytes beeing sent for the CCs could propably be optimized by using the "running status" feature. Instead of sending "B0 mm nn B0 oo pp B0 qq rr", you can also send "B0 mm nn oo pp qq rr" (the last status byte will stay active)

But in general the MIDI interface of your synth is the bottleneck, it's not possible to increase the baudrate here, thats the problem

The USB interface cannot be serviced from a PIC, since a "USB host interface" + appr. drivers are required.

A general question: do you always send dumps and CCs on button movements, regardless if they already have been sent with the same content/values? Maybe you could optimize here - only send values which have been changed.

I don't see much more possibilities for optimisation here w/o using a PC as USB host which works as a bridge between the PIC and the synth,

Best Regards, Thorsten.

[robinfawell]

Dear Thorsten

I was replying to you explaining that I thought it was the length of the sysex messages when you sent the second reply.

Debounce is not specified.

Only one sysex message will be set with each press of a button.  There are no multiple dumps. The  CC's are very short.  I dont think that they are a problem. Anyway the majority of CC's are volume, balance or expression values from pots.  The coupling controls only change the channel numbers of the note on/off messages.

On most stops there is at least 2 seconds delay before the synth will play notes.  This is about a factor of 10 compared to the 0.32mS figure. (I think you mean  0.32 seconds)

Could the Sound Canvas module delay the process?  I cannot see any reference in the manual.

Regards Robin

[TK.]

Hi Robin,

I'm not sure, "2 second delay" sounds like a MPROC timeout. Before we continue the discussion, could you please fill the NotifyTimeout function in this way:

void MPROC_NotifyTimeout(void) __wparam
{
   MIOS_LCD_CursorSet(0x00);
   MIOS_LCD_Clear();
   MIOS_LCD_PrintCString("TimeOut!");
   MIOS_LCD_MessageStart(255);
}
[/code]
Do you see the Time Out message on the LCD when the delay happens?
Best Regards, Thorsten.

[robinfawell]

Dear Thorsten

I included the suggested code firstly in Core 1.  No "Timeout"

With the code in Core 2 "Timeout" occurred in some circumstances but not others.

Before the details are given I should explain that the stops and the pedal notes are on Core 1.  This connects to Core 2 with a cable not Midi to Core 2.  The two Keyboards are on Core 2 which is connected to the synth by midi out.

1) There is no "Timeout" with the Pedal stops, even with one stop "Octave 8" at 2472 bytes and the others at 1236 bytes.

2) There is no "Timeout" with 4 Stops.  These are however, fairly short at 756 bytes.

3) I have "Timeout" on all other stops.  These have different file lengths mainly about 1250, a few just under 1000 and other at about 1400

There is an anomaly with the  2 Cancel sysex at 652 bytes that cause "Timeouts" even though they are smaller than the 4 stops in 2) at 756 bytes.

The above results gave me a thought.  The most midi activity occurs when the sysex messages are sent.  Therefore it makes sense to connect the busy

core module to the synth.  This is not very scientific, I know.

The result is that there is a vast improvement.  There are no "Timeout" messages.  There is an interruption of the notes for a fraction of a second but not for 2 or 3 seconds.

I am curious to know the reason for the "Timeout" test  and more importantly why the system works better with the "stops" module connected to the sound module, rather than the "notes" module.

Thanks again

Robin

[robinfawell]

Dear Paul

Perhaps it was not explicit in my reply to Thorsten but I am delighted with the current state of the project.

I now have a working design.

I still need to tidy up the programming.  The memorised setup program is yet to do and I plan to make a new console with engraved legends.

The basic electronic design and program is complete.

Regards Robin

[TK.]

Hi Robin,

I'm relieved that this wasn't a conceptional issue (just thought about how many time you've already worked on it)

A timeout can happen if the received message from a MIDI device is not complete. The MIDI stream processor always waits for a the end of a MIDI event before it exits and the main task is continued. So long the last byte of a MIDI event haven't been received, it stays in a time out loop, which will be terminated after 2 seconds to notify a malfunction of the MIDI sender

So - it makes sense to check the code of your "SysEx core" if it could send incomplete SysEx streams or other events. Unfortunately a MIDI monitor like MIDI-Ox cannot help so much for debugging here, because it has the same "problem" like any other MIDI processor - it waits for complete events before they will be displayed.

Therefore it's maybe better to temporary disable parts of the MIDI sending routines until you find out, which one causes the timeout

Best Regards, Thorsten.

[greenfox]

Glad to hear it is working now for you Robin.  I will try my 2x cores in different order to see if it reduces slight delay I find in my pedals.

Have you tried Miditzer? http://www.virtualorgan.com/  It uses the PC to generate the sounds and has a much lower trafic rate through Midibox.

[robinfawell]

This reply is to KealyPaul and Greenfox

The overiding design philosophy was to remove the PC from the SCPOP computer based design.  This organ will eventually return to the local church to be played by the organist.  As designed, the SCPOP utilised a display, keyboard and mouse.  These items are not appropriate for the church and are not necessary.  Therefore any software sysem eg jorgan miditser are not compatible in a simple way (I believe!)

What I have done is to reduce the number of stops and other controls and replicate the SCPOP sysex messages and other Midi CC's to the SC.  (In my case the SC 8850).

You can read the SCPOP web site although now it is unsupported. The system uses thee midi channels for the upper, lower and pedal sounds.  The stops for the upper and lower manuals are each in 3 Radio groups.  ie only one stop from each group  can be selected at a time.

The individual stop sounds are layered and are a mixture of the SC Midi .  The sounds utilise the SC Parts as follows:-

Lower Manual parts 1-6, Midi Channel 1

Upper Manual parts 7-12, Midi Channel 2

Pedals parts 13-16, Midi Channel 3

In my case I have utilised core 1 for the 2 keyboards and Core 2 for the Console stops and controls and the pedal notes.  The C program directs the various sysex messages that are stored in separate EEprom (Banksticks), to the Midi Out.  There are a number of additional controls associated with tremelo an coupling stops.  Core 1 and 2 have analog inputs which control the balance of the radio group outputs and depth of tremelo.  I plan to have 5 or 6 preset memories that store favourite stop and balance control settings combinations.

By the way the Radio Groups are :-

Principals and Flutes

Mixtures and Mutations

Reeds and Strings.

This is as you see, a stand alone system, No pc!

It is not based on Midio 128 and is my software design with a lot of help from Thorsten.

Regards Robin

,

[robinfawell]

Reply to Thorsten

Thanks for the explanation. I do not understand all the hardware and software design aspects  sufficiently to offer any proper alternative reasons for the problem.

I would love to delve further into the reasons why the original setup with core 1 for console and core 2 for keys has a timeout problem and the opposite has no timeout problem.  However I am very late with this project already.

You may recall that with my original setup there was no timeout on the "console" core.  The timeout was on the "notes" core.

In a simple-minded way I believe that the midi activity is so high when sysex messages are being sent from the console core to the notes core that the notes from the keyboard cause a malfunction in its core.

The program for the "notes" core is very basic.  There is the normal Note On/Off  code, there are two analog pots for controlling signal values.

One thought concerns the fact that the cores are not synchronised in any way.  I am using a cable to direct the midi and not using the Midi out from the first core.  I have another cable from the console connecting a Dout to a Din on the "notes" core.

Although the current system works fairly satisfactorily there is no doubt that when the stops are pressed whilst notes are played there is an interruption.  I would guess that during the 0.3 sec the notes are ignored or corrupted.  However I think that is tolerable.  I plan soon to have a visit by a well experienced organist to take a critical view of the complete system.

Regards Robin

PS Sent Later

I have tested the original SCPOP PC based system.  It has  similar characteristics as my current setup. When changing stops when playing there is some discontinuity in the SCPOP case it is loss of notes. 

The main problem would be hung notes. I have not experienced this yet with my design.