I need to scan the SIO every 10us, can it be done?

[jmayes]

Hi all, I am currently working on a matrix scan system using the DIO128 w/64analog example application. I have it working fine at 1ms intervals but to match the scan system of my organ I need it to operate at 10us, that is I need to shift the first 64 bits of the SIO within that amount of time, I can then wait 10us between scans. Can it be done? MIOS32 perhaps?

Please someone let me know if it is at least possible so I can decide if I need to go back to plan-b.

Thankx all,

jmayes

[TK.]

10 uS scan cycle for 64bit means that the required scan frequency is ca. 6.5 MHz.

This can't be done with the PIC (if other tasks should run in parallel), but it's possible w/o much programming effort with MIOS32, because the DMA could fetch the shift registers in background while the CPU processes the results of the previous scan in parallel.

Best Regards, Thorsten.

[jmayes]

10 uS scan cycle for 64bit means that the required scan frequency is ca. 6.5 MHz.

This can't be done with the PIC (if other tasks should run in parallel), but it's possible w/o much programming effort with MIOS32, because the DMA could fetch the shift registers in background while the CPU processes the results of the previous scan in parallel.

Best Regards, Thorsten.

Thank you TK for the fast concise answer. I guess it's time for me to invest in a few MIOS32 cores to continue on my quest. Will I be able to use the C app I have already written and just dink with the DMA settings or will it require some work in the root mios asm?

Thankx again!

Jmayes

[TK.]

With MIOS32 we don't program in assembly language anymore; C programs written for MIOS8 are not 1:1 compatible, but have a similar look and feel.

See also: http://www.ucapps.de/mios32_c.html

The README files describe the new concepts in detail.

MIOS32 already uses DMA for SRIO transfers, this article will give you some additional background informations for your "mission":

Since your application is even more timing critical than we one described in, you could also disable the normal SRIO handling, and call MIOS32_SPI_TransferBlock directly.

See also http://www.midibox.org/mios32/manual/group___m_i_o_s32___s_p_i.html

Best Regards, Thorsten.

[jmayes]

Thankx again TK!

Yes, I was just coming to the realization that the o'l mios8 programs would not load into mios32 but from what I have figured out so far the routines I have already made will work with little modification. The most help I will need will be to get it configured for the high speed transfers. I have ordered a few from SmashTV to play with and I will look up the info you supplied. Hopefully by the time they arrive I will have a battle plan.

Thankx again!

Jmayes

[jmayes]

Ok, I have my Core32 tested and have my app (mostly) converted. I think I can figure out how to use the user-timer to trigger the transfers but can't quite wrap my head around the DMA thing. Would it be possible to get some more info or some example code?

Thankx again for the help,

Jmayes

[jmayes]

I have my app working fully but it is still not fast enough. I am triggering the SRIO_start with the user_timer and I found that changing the prescalier value in the SRIO.C module would speed up the transfer to around 15us. Anything faster and the 5v pullups would fail. Now my biggest problem is processing the data after the xfer is too slow which at present is 150us. I also found that adding my code to the SRIO.C was way faster then in app.c so I am also doing that. Now I think I need to process data as the SRIO shift reg's are updating (in the same routine) in order to get to the next faster plateau??

Questions?

Exactly where is the low level SRIO xfer code?

Is there a better way?

After more testing I found I can go as slow as 25us (including scan) and still achieve my goal.

Thankx again for any help someone can provide,

Jmayes

[TK.]

Hi Jmayes,

the SRIO clock frequency can be increased by configuring the J8/J9 pins for push-pull mode, and by using a 74HCT541 buffer as shown in this schematic: http://www.ucapps.de/mbhp/mbhp_core_lpc17.pdf

Although this circuit is intended for the upcoming MBHP_CORE_LPC17 module, it will work for STM32 as well.

Important: you have to add

#define MIOS32_SRIO_OUTPUTS_OD 0

to your mios32_config.h file

Exactly where is the low level SRIO xfer code?

MIOS32_SRIO routines are located in mios32_srio.c

MIOS32_SPI routines for STM32 are located in mios32_spi.c

Best Regards, Thorsten.

[jmayes]

Have you ever seen the light at the end of a tunnel and just can't quite get to the end? I am almost there with my project but still there is one last road block (I hope the last one).

Ok, on my quest to scan the buss on my Rodgers organ which must be done every 10us I have moved to the Core32, I have put the SRIO scanning into a tight loop so that the SR_Finish directly restarts the SRIO_SCAN. I use several mux pins (GPIO) that output between scan cycles. There are 4 scan cycles then it repeats. I have this working with J5A perfectly all the way to a pre-scalier of 8 which is getting my cycle time down to 17us. The final thing I needed to do is to add some additional GPIO outputs (4 more) in order to trigger the tone generators separately from the keyboards. Just a slight variation of what J5A pins are already doing.

Now the weirdness, once I started using J5B pins I get no Dout. The scanning is working fine but there is just no data going to the SR's. Additionally when it does this the midi port will no longer allow MIOS studio to query it and I can't download without cycling power and letting the bootloader take over. Din is still working, the J5 pins (all 8 of them) are working as expected but there is just no Dout.

I spent many hours putting rem statements in front of my code to see what was causing this. It came down to I can use EITHER the 4 pins of J5A OR 4 pins of J5B but NOT BOTH!! I could not find anything at all in my program that seemed wrong. Finally I changed the pre-scalier to 16 and boom, everything is working perfect (but 28us is too slow for my app). That proves to me that my code is ok. I was really hoping to eventually go to a pre-scalier of 4 in the final rev. (takes me to 12.5us cycle)

I was using the OD option and 1k pull up resistors, along the way I tried the PP and no resistors and got the Dout to work some with all 8 pins but it was delayed??? It took like around 3-4 ms from when I sent midi to when the Dout would output, all I had to do is rem out J5A pins or J5B pins and it would start working perfectly again??? Also I am using a scope to monitor the Dout to the SR's to confirm there is no data outputting. I am using buffers on the clock, latch and data and I do have din and dout working down to pre-scalier 4 just not with 8 J5 pins.

I will attempt to use J19 pins to get around this but I thought perhaps TK or another expert user might be able to put some light into my problem. I AM SO CLOSE!

Here is the code In question, I put it in MIOS32_SRIO.C

...........................

static void MIOS32_SRIO_DMA_Callback(void)

{

// notify that new values have been transfered

srio_values_transfered = 1;

MIOS32_BOARD_J5_PinSet(muxctr , 0); //jm organ muxpin off (J5A)

MIOS32_BOARD_J5_PinSet(muxctr + 4 , 1); //jm keyboard mux pin on (J5B)

//latch DOUT registers by pulsing RCLK: 1->0->1

MIOS32_SPI_RC_PinSet(MIOS32_SRIO_SPI, MIOS32_SRIO_SPI_RC_PIN, 0); // spi, rc_pin, pin_value

//MIOS32_DELAY_Wait_uS(1);

MIOS32_SPI_RC_PinSet(MIOS32_SRIO_SPI, MIOS32_SRIO_SPI_RC_PIN, 1); // spi, rc_pin, pin_value

MIOS32_BOARD_J5_PinSet(muxctr + 4 , 0); //jm keyboard mux pin off (J5B)

//JM Mux counter

if ( ++muxctr >= 4 ) {

muxctr = 0;

}

//mux pin on

MIOS32_BOARD_J5_PinSet(muxctr , 1); //jm organ mux pin on (J5A)

.................

Here are my MIOS32_Config.h def's

#define MIOS32_DONT_SERVICE_SRIO_SCAN 1 //jm

#define MIOS32_SRIO_NUM_SR 8 //jm

#define MIOS32_DONT_USE_ENC 0 //jm

#define MIOS32_DONT_USE_DIN 0 //jm

#define MIOS32_DONT_USE_AIN 0 //jm

#define MIOS32_DONT_USE_COM 0 //jm

#define MIOS32_SRIO_OUTPUTS_OD 0 //jm

Thankx again for any help someone can provide,

Jmayes

[TK.]

Hi Jmayes,

I guess that the interrupt routine consumes too many CPU cycles, so that there isn't enough time to handle other threads, such as USB.

But there is much room for optimisation - could you please attach you project in a .zip file, so that I can have a look?

(please type "make cleanall" before to remove the binaries)

Best Regards, Thorsten.

[jmayes]

Thank you TK for taking a look at this for me!!!

Here are the only files that I have changed from the "Traditional" mios32 model, I have marked //JM on

most lines That I have added or changed. This file set runs exactly correct on the J5 pins w/din and dout at pre-16 but when you change it to 8 the dout stops. Your comment leads me to believe that it is the midi in routine on uart0 what is failing, that would explain why I loose download ability too. (I do not use USB)

Program description:

Basically I have made all din and dout buffers 2 dimensional using the "muxctr" and "level" vars. I did

also modify all the debounce and din handler code but now am not using it as I have added a new Din

handler to the app.c using the user-timer to trigger it every 800us. Each scan is at a different mux

level that repeats after 4. This allows for 3 manuals + Pedals (4 levels). Midi is input and output in

4 sequential channels starting at ch1 on uart0 (Midi port 1)

I do not use USB at all, we can disable it if it will help. The biggest issue (other then the J5 pin

weirdness) is the 7us before the SRIO scan and the 3us afterward. If I could reduce that, then I could

run at a 16 pre-scalier. See 17us-scan1.jpg, note I took that shot with the pre-scalier at 8.

Thankx again for looking over this for me,

Any suggestions will be appreciated!

Jmayes

V13-outfile.zip

Edited May 21, 2011 by jmayes

[jmayes]

Solved!!!

Turns out the 800us I was triggering my custom DIN handler routine became too short (and caused overlaps) after I added the instructions for the extra pins I needed. Changing to 1ms has fixed all! Still can't explain why it caused a dout problem but at least I know what did it and can move on.

I still need some help knocking the dead space down between SRIO scans if someone can help.

Thankx again,

Jmayes

[TK.]

I'm going to write a demo application for you and already optimized a lot of things.

More infos in ca. 30 minutes

Best Regards, Thorsten.

[TK.]

Hm... maybe it's better to give you a half-ready demo for discussion, because some points are not clear to me.

You will find the sources under http://www.ucapps.de/tmp/fast_srio_scan_v0_1.zip

Optimizations:

• instead of patching code into original MIOS32 routines, I started to copy the relevant SRIO/DIN/DOUT parts into app.c - typically this gives more freedom and a better oversight (use MIOS32 code only as a template)
  
• the SRIO_Handler prepares the next SRIO transfer, when it's finished the DMA IRQ will call SRIO_Handler again -> it's triggered periodically
  
• the RC pin is only toggled once between the transfers - thats a trick! While it captures new DIN values, it also captures the DOUT values at the same moment.
  
• I optimized some bits of mios32_spi.c, please update the repository
  
• gap between SRIO transfers: only ca. 5 uS anymore (but will be a bit longer once DIN values are stored before next transfer)
  
• SPI pins configured for Push-Pull with *strong* drivers (should be faster than weak drivers)
  
• J5 pin accesses now done directly via SFR write operations (less readable code, but in total the SRIO_Handler runs ca. 2 uS faster)
  
• prepared TASK_DIN_Handler which is running in background and which will check for pin changes in an endless loop. It can be interrupted by higher-prio MIOS32 tasks. Using a task instead of an interrupt routine makes much more sense here, as MIDI transfers could be blocking (e.g. if UART fifo is full) - maybe you already noticed this effect - interrupts should be executed as fast as possible and they shouldn't block for more than 10 uS (maximum 100 uS)
  

Although the DIN handling (check for pin changes) hasn't been implemented yet (I can do this once the approach is clear), I guess that the typical scan speed for 4*8 SRs will be ca. 200 uS if SRs are clocked with 16x prescaler

Now my question: why do you need such a high scan speed, although you are only using a common UART to send MIDI values?

Sending a MIDI event which consists of 3 bytes takes 960 uS, by using running status optimization (channel number doesn't change) 640 uS

So, does it really make sense to scan faster?

Due to the reduced MIDI speed (as long as you are not using USB) you wouldn't notice an effect.

Or wouldn't it make more sense to slow down SRIO for more robustness, and/or use common MIOS32 access functions (e.g. for J5) to make the source code more readable?

Best Regards, Thorsten.

[jmayes]

I'm going to write a demo application for you and already optimized a lot of things.

More infos in ca. 30 minutes

Best Regards, Thorsten.

That will be really great! Looks like I am hitting a wall now. Pre-8 is the smallest I can use before the hardware chain fails, I am using strong-outputs and the buffer you suggested as well as terminators at the end of the chain. Also I did some scoping and my din handler which takes 700us at pre-16 ends up taking 1500us at pre-4 so no wonder things were getting mucked up as multiple threads were overlapping. Sitting at 20us now, almost usable. It creates a high pitch whistle on the organ that only my young nephew can hear.

Thankx again!

Jmayes

[jmayes]

Looks like we were posting at the same time! I will look over what you have, that optimized spi.c might be just the extra I need.

The reason I have to scan so fast is that the organ generator system multiplex's all 4 divisions on a single 61 note bus (time division). Since the keyers are analog if the single division mux pulse is slower then 12us it translates to something that can be heard (Hi pitch) when keys are played. They choose 10us which turns out to be barely under perfect. (hear-able pitch is above 20khz). I could slow it down and put a bunch of caps on all the generator outputs to snub freqs over 18k (or 16k) but that will lead to alot of extra work on the organ. My goal is to be able to do the upgrade with the least amount of mods to the organ itself.

I am real close now, it's all working with about a 18k squeal (which I personally can not hear).

Going to start going over what you sent.

Thankx again!

Jeff

[TK.]

Ok, it's clear now.

So, I added the DIN handling: http://www.ucapps.de/tmp/fast_srio_scan_v0_2.zip

Thats the last change I've planned to make - I haven't checked if DIN values are handled correctly, but as mentioned: it's just a demo for possible optimizations.

8 SRs are scanned in ca. 8..9 uS. The period is slightly jittering by ca. 1 uS, but this should be acceptable. It never takes longer than 10 uS

The DIN_Handler calls APP_DIN_NotifyToggle with following parameters:

	    u32 pin = MIOS32_SRIO_NUM_SR*8*mux + 8*sr + sr_pin;

	    u32 pin_value = (srio_din[mux][sr] & (1 << sr_pin)) ? 1 : 0;

	    APP_DIN_NotifyToggle(pin, pin_value);

[/code]



You have to decode the pin number again in this hook, or alternatively define another hook (if this is easier for you) which has the mux number as additional parameter.


DOUT pins have to be set/cleared by writing directly into the srio_dout[mux][sr] array.


Hope it works!


Best Regards, Thorsten.

[jmayes]

TK, YOU ARE THE MAN!!!

This is exactly what I needed, I will buy you 2 bears- heck a keg if there is an option!

Just a few minor things, I was able to get dout working but the din handler does not seem to be scanning. I don't think it's being called at all. Also that part where you did not optimize where it looked like a test is really needed, J5-11(input) changes it from mode1 to mode2 where the keyboards will be separated from the organ generators. (For some reason J5-11 input is not working). I also use J5-8 output which I am going to use for hardware blanking of the dout so when I send midi it won't loop back on the din's.

Here is my (almost final) app

Thankx again!

Jmayes

Rodgers-App2.zip

Edited May 22, 2011 by jmayes

[TK.]

I'm glad that it's (almost) working!

And thanks for the beers! :)

Yes, the DIN buffer copy routine has a typical copy&paste error - it copies mios32_srio_* variables, although app.c uses local variables called srio_*

Update is here: http://www.ucapps.de/tmp/fast_srio_scan_v0_3.zip

Changes: before muxctr is incremented, I store the previous counter value in a temporary variable:

  u8 prev_muxctr = muxctr;

  if ( ++muxctr >= 4 ) {

        muxctr = 0; 

  }  

[/code]




DIN buffer copy and change detection:

[code]

  // copy/or buffered DIN values/changed flags of previous scan

  int i;

  for(i=0; i<MIOS32_SRIO_NUM_SR; ++i) {

    srio_din_changed[prev_muxctr][i] |= srio_din[prev_muxctr][i] ^ srio_din_buffer[prev_muxctr][i];

    srio_din[prev_muxctr][i] = srio_din_buffer[prev_muxctr][i];

  }

I've no explanation why J5 input #11 doesn't work - did you replace the "#if 0" by "#if 1" to enable the if condition?

#if 1

  if (MIOS32_BOARD_J5_PinGet(11) == 0) {        //test for mode 1

      MIOS32_BOARD_J5_PinSet(muxctr + 4 , 1);   //jm set keyboard mux pin on mode 1

  }

#endif

[/code]


Best Regards, Thorsten.

[jmayes]

Ok, we are so close now I can taste it! (or at least drink it...)

Corrected!

I am getting DIN's now but with some strangeness. When I press pin-0, Mux-0 I get pin 40, pin-0 on mux-1 gives nothing (out of range?), mux-2 gives C0 and Mux-3 gives 00 (correct).

I had to add the mux var to APP_DIN_NotifyToggle in order to pass it from your din handler.

Dout is perfect!

Time has gone up to 12.5us but that is fine, no need to optimize further.

Oh, Ya I did not even see you put an IF around my (test) using J5-11, that fixed it (Duh!)

Here's the app as it stands now.

Thankx again!

Jmayes

Rodgers-App2.1a.zip

Edited May 22, 2011 by jmayes

[jmayes]

Solved!!!

Everything is fully up at running at 12.5us

In the new Din Handler the way the pin was calculated was wrong;

Was:

u32 pin = MIOS32_SRIO_NUM_SR*8*mux + 8*sr + sr_pin;

Now:

u32 pin = 8*sr + sr_pin;

That made it all good :)

Once I polish the app a little more I plan on starting a new thread in midification with full details on how to implement this.

Thankx again TK, your the greatest!

Jmayes

[jmayes]

..

Edited May 23, 2011 by jmayes

[TK.]

Cool! :)

I merged the topics and moved the article to MIOS Programming

u32 pin = 8*sr + sr_pin;

This means that the same pin number is passed for different multiplexer groups, is this ok for you?

It could make sense to forward the mux variable to the function which handles the buttons (if required), e.g. if you want to send Note events over different channels.

Best Regards, Thorsten.

[jmayes]

This means that the same pin number is passed for different multiplexer groups, is this ok for you?

It could make sense to forward the mux variable to the function which handles the buttons (if required), e.g. if you want to send Note events over different channels.

Yes, that is exactly what I am doing, each mux level has it's own midi channel.

Thankx again for all your help!

Jmayes

[jmayes]

Got it installed in the organ with a few (minor)issues, one hardware (more on that later) and one software. TK, is there anyway I could get you to add de-bounce to the din handler? Turns out that at this ultra-fast update cycle I am getting lots of extra traffic on the midi. I think 3-4 din handler scans before triggering would be about right.

Attached is my (almost finished app). I have added midi key offset and mux/midi correction so that din's and dout's are on the same mux level and at the right octave. Overall I am very excited, not far to go now!!

Here are a few pix of how the project is coming.

Thankx again for the help!

Jmayes

Rodgers-App2.3a.zip

Edited May 25, 2011 by jmayes