SRIO, DOUT, DIN, 16 bit , 8bit Confusion

[Duggle]

This also raises the subject of DMA. I think that it would be interesting to see what is possible and would add to Duggle's work on this subject.

I'd encourage you now that you have your application working to enhance it and extend it in various ways that occur to you as useful, keeping the variously modified working versions for reference.(Its the best way to learn!)

Having said this Interestingly if 226ms or so is the lowest velocity then a 1ms sample rate seems actually more than adequate. How is the velocity resolution for fast (or rather, hard) key hits? I imagine it is very good.

If you were to implement a range of selectable velocity response curves then its conceivable that you could do with the raw velocity timings of a higher resolution (implying a faster scan rate). I suspect this to be only marginally subjectively beneficial.

With short wires and few DIN and DOUT, I don't think signal integrity issues will stop you increasing the spi clock, allowing you to increase the scan rate.

[robinfawell]

Duggle

Having said this Interestingly if 226ms or so is the lowest velocity then a 1ms sample rate seems actually more than adequate. How is the velocity resolution for fast (or rather, hard) key hits? I imagine it is very good.

I must admit that the general impression of the sound and playability is good. Actually getting a velocity of 1 needs great attention.

Not much force is needed to achieve a velocity of 100. The may be some scope for greater resolution above this level. A light touch can easily produce a velocity of 40 - 60. It is very subjective!

Robin

Edited October 7, 2010 by robinfawell

[TK.]

Hi Robin,

I'm glad that the code is working, although I never had the chance to test it by myself.

Your descriptions and comments made it easy to follow the progress remotely! :)

Using different methods (e.g. faster scan speed, faster update cycle, using DMA) will lead to a lot of extra work for that I think it isn't really worth the effort!

From my experience I can already say that this won't improve the results, and Duggle confirmed this in his comment.

By scanning the complete matrix within 1 mS you've already a very optimized version, with common MIOS32 SRIO driver it would normaly take 16 mS (since a single DIN pair of the matrix would be scanned each mS) which is slower than the shortest possible delay.

Now you've a version which scans faster than the shortest delay.

To your table: could you please also count the pin numbers from 0 instead from 1?

This would simplify the documentation as well, because currently I had to add some comments into the source code to explain the difference, which might be confusing.

Starting note: ok, will be changed to 0x15

Velocity curves: good point!

It could make sense to apply a logarithmic curve, so that short delays lead to higher differences than long delays - I will check later today (or tomorrow) if it's possible to calculate the curve during runtime instead of using a pregenerated table to allow finetuning w/o recompilation.

Best Regards, Thorsten.

[robinfawell]

Thanks Thorsten

I accept your points regarding speed.

I suppose when you think about some musical instruments, there are always delays eg on an acoustic piano: fingers to keys, keys to hammers, hammers to strings. With light touch notes the hammer and key action is of course slower. It could be regarded as more realistic having longer delays.

The table is already revised . It seemed better to use a generally accepted rule regarding bits, pins, SR pins. I have expanded the document to include more details of the Fatar connections. See below.

Regarding different responses eg logarithmic. Come soft synth pianos have different curves built in. The one I'm using "Pianissimo" has several curves from very light to very heavy. I have just found this out!

Best Regards Robin

Row Col Fast Scan pin from 0.doc

Edited October 7, 2010 by robinfawell

[Duggle]

I accept your points regarding speed.

I suppose when you think about some musical instruments, there are always delays eg on an acoustic piano: fingers to keys, keys to hammers, hammers to strings. With light touch notes the hammer and key action is of course slower. It could be regarded as more realistic having longer delays.

I agree with your observation but bear in mind:

The comments made about scan speed only relate to the resolution of raw timing values that get converted into midi velocity from a possible 127 down to 1. You already have nearly double the resolution you require for the linear response curve!

Now when we talk about delays in this context it's usually referred to as latency, its the speed with which events manifest from a cause.

In modern DAW's and digital instruments etc often the greatest latency is caused by the distance from the ear to the monitor speaker (speed of sound 1 millisecond per foot!)

Other sources of latency (that get added together) are MIOS32 event processing cycle (1ms, I think), uart MIDI (1ms per note), or USB to host system (about 1ms or more with jitter), DAW host and softsynth (<=5ms very good, 10ms or so is usually acceptable)

[TK.]

I updated the files so that they are matching with your pin table.

And I copied the .doc file into the doc/ folder.

Probably you have to change some matrix connections on your hardware if this hasn't been done yet.

To determine a suitable logarithmic curve it would be interesting which delay you measure for different pressures, e.g. "hard", "normal", "semi", "soft", "very soft" - and which velocity values should be output in such cases.

Best Regards, Thorsten.

[robinfawell]

Dear Thorsten

There is an error. Here are the first few notes.

[2864.223] row=0x01, column=0x00, pin_value=0 -> pin=1, timestamp=12411 -> IGNORE

[2864.230] row=0x02, column=0x00, pin_value=0 -> pin=16, timestamp=12418 -> IGNORE

[2864.356] row=0x02, column=0x00, pin_value=1 -> pin=16, timestamp=12544 -> NOTE OFF

[2864.373] row=0x01, column=0x00, pin_value=1 -> pin=1, timestamp=12561 -> IGNORE


[2870.294] row=0x01, column=0x01, pin_value=0 -> pin=3, timestamp=18481 -> IGNORE

[2870.303] row=0x02, column=0x01, pin_value=0 -> pin=18, timestamp=18490 -> IGNORE

[2870.415] row=0x02, column=0x01, pin_value=1 -> pin=18, timestamp=18602 -> NOTE OFF

[2870.429] row=0x01, column=0x01, pin_value=1 -> pin=3, timestamp=18616 -> IGNORE


[2879.430] row=0x01, column=0x02, pin_value=0 -> pin=5, timestamp=27616 -> IGNORE

[2879.435] row=0x02, column=0x02, pin_value=0 -> pin=20, timestamp=27621 -> IGNORE

[2879.540] row=0x02, column=0x02, pin_value=1 -> pin=20, timestamp=27726 -> NOTE OFF

[2879.552] row=0x01, column=0x02, pin_value=1 -> pin=5, timestamp=27738 -> IGNORE


[2884.359] row=0x01, column=0x04, pin_value=0 -> pin=9, timestamp=32544 -> IGNORE

[2884.367] row=0x02, column=0x04, pin_value=0 -> pin=24, timestamp=32552 -> IGNORE

[2884.479] row=0x02, column=0x04, pin_value=1 -> pin=24, timestamp=32664 -> NOTE OFF

[2884.494] row=0x01, column=0x04, pin_value=1 -> pin=9, timestamp=32679 -> IGNORE

I have also looked at the util doc file, this starts with pin = 1 Also look at my table it starts with Pin No = 0. I can do this again but I thought this is what you wanted. Row, Column, Pin No all starting with "0". There is a less important comments error, referring to BLM.

/////////////////////////////////////////////////////////////////////////////

// This task is called each mS to scan the button matrix

/////////////////////////////////////////////////////////////////////////////

// will be called on BLM pin changes (see TASK_BLM_Check)

void BUTTON_NotifyToggle(u8 row, u8 column, u8 pin_value, u32 timestamp)

I will run the key touch tests later today.

Robin

[TK.]

Hi Robin,

as mentioned earlier, you will have to change your hardware connections to get the pin numbers matching again.

Previously the first pin was in row 1, although the software already started to count from row=0 (so, it was the second row)

Just check your DOUT connections, all pins have to be shifted by one.

Alternatively try following change:

// will be called on button pin changes (see TASK_BLM_Check)

void BUTTON_NotifyToggle(u8 row, u8 column, u8 pin_value, u32 timestamp)

{

  row -= 1; // we don't know why rows are starting from 1 on Robin's Hardware

[/code]


Best Regards, Thorsten.

[robinfawell]

Dear Thosten

We may be at cross purposes here.

First of all let us agree some terminology. Let us agree that all pins, rows, columns and SR pins (bit x) begin at 0.

My hardware is currently connected as follows:-

J3 DIN (bit 0 of SR) to Col 0, J3 DIN (bit 1 of SR) to Col 1, etc, ending with J3 DIN (bit 7 of SR) to Col 7

J4 DIN (bit 0 of SR) to Col 8, J4 DIN (bit 1 of SR) to Col 9, etc, ending with J4 DIN (bit 7 of SR) to Col F

J3 DOUT (bit 0 of SR) to Row 0, J3 DOUT (bit 1 of SR) to Row 1, etc, ending with J3 DOUT (bit 7 of SR) to Row 7

J4 DOUT (bit 0 of SR) to Row 8, J3 DOUT (bit 1 of SR) to Row 9, etc, ending with J3 DOUT (bit 3 of SR) to Row B

/

row -= 1; // we don't know why rows are starting from 1 on Robin's Hardware

My rows do start at 0 (not1)

Originally I think that I referred to J3 DOUT bit 0 as Row 1. This I'm sure is the cause of confusion.

I'm sorry to be a nuisance.

Robin

Edited October 8, 2010 by robinfawell

[robinfawell]

Dear Thorsten

I have checked the DOUT connections using a combination of MIOS STUDIO and a meter on the "diode" setting.

MIOS STUDIO was very useful in showing when the depressing of the key made only the 1st switch close .

I used weights from the kitchen to rest on the first key (A0) to make only the the 1st switch close.

I have shown that J3 DIN IC pin 11 (D0) equiv to T0 (Fatar)is connected to J3 DOUT IC pin15 (D0) equiv to BR0 (Fatar) (via a diode) when the key is depressed making the 1st switch of the pair close only. In this state there is no connection to pin 1 to J3 DOUT IC pin 1 (D1) equiv to MK0 (Fatar).

When the key is fully depressed then J3 DIN pin 11 (D0) is connected to both DOUT pins ie D0 (BR0 and D1 (MK0)

I believe that this shows that the first row is row 0 and that my wiring is OK.

By the way the expression

row -= 1;

corrects the pin No(s) to agree with my table. In trying the find out the reason I wondered about

void BUTTON_NotifyToggle(u8 row, u8 column, u8 pin_value, u32 timestamp)

{

  // determine pin number based on row/column

  // based on pin map for fadar keyboard provided by Robin (see doc/ directory)

  // tested with utils/test_pinmap.pl


  int pin = -1;

I could not understand why the variable pin was given a value of -1. I tried

int pin;

and there seems be be no difference to the system. If I am right the it follows that something else is wrong. I know that it doesn't matter because the piano can be made to work.. Maybe it is something to do with the operation of DIN/DOUT in latching the previous DIN inputs. Code alternative I don't understand

  // pin number (counted from 0) consists of:

  //   bit #0 if row -> pin bit #0

  int bit0 = row & 1;

  //   bit #2..0 of column -> pin bit #3..1

  int bit3to1 = column & 0x7;

  //   bit #3..1 of row -> pin bit #6..4

  int bit6to4 = (row & 0xe) >> 1;


  // combine to pin value

  if( column < 8 ) {

    // left half

    if( row >= 0 && row <= 0x9 ) {

      pin = bit0 | (bit6to4 << 4) | (bit3to1 << 1);

    }

  } else {

    // right half


    if( row >= 0 && row <= 0xb ) {

      pin = 80 + (bit0 | (bit6to4 << 4) | (bit3to1

As an exercise I have worked out an alternative that I understand.

	// for the left hand, pin = 8 * row + 2* column when row even, pin = 8 *(row - 1) + 2 * column +1 when row odd

	// if ( row % 2 == 0 ) { it's even} else{ it's odd}	

	//left half, column < 8, row 0-9),  pin even


	if ((column <0x8) && ( row >= 0 && row <= 0x9 ) && (row % 2 == 0))

	{pin = 8 * row + 2* column;} else{

	if ((column < 8) && ( row >= 0 && row <= 0x9 ) && (row % 2 == 1))//is odd

	pin =  8 *(row - 1) + (2 * column) +1;}	


	// right half,(column (8-F),row (0-b), pin even, in the right half need to add 80 (even)or 81 (odd)

	//and subtract 8 from column value to get correct pin nos


	if((column >= 0x8 && column <= 0xf) && (row >=0 && row <= 0xb) && (row % 2 == 0))

	{pin = 80 + 8 * row + 2* (column - 8);}else{//row is odd

	if((column >= 0x8 && column <= 0xf) && (row >=0 && row <= 0xb) && (row % 2 == 1))//even rows

Touch experiment

You asked me to undertake an experiment with various degrees of touch to the keys. Here are the results , highly subjective of course!

Hard 126

Normal 100 - 105

Semi 80 - 90

soft 50-60

very soft 1

Finally, having thought about the scan speed issue, the only in advantage in increasing speed would be that of time accuracy. ie from a 1mS accuracy to 0.5mS accuracy.

I doubt whether the switch contacts are mechanically displaced all that accurately.

Best Wishes Robin

Edited October 10, 2010 by robinfawell

[TK.]

Hi Robin,

I think that I found the reason why the row index started from 1 instead of 0.

Before the first row was scanned, the selection pattern for row=0 was shifted into the DOUT registers, but it hasn't been latched.

Accordingly, the first DIN scan started with the selection pattern for the last row (row=15, which is not connected at your side)

Now I'm pulsing RC before the scan is started.

At the beginning we will have two pulses, one to latch DOUT values, another to latch the DIN values of the selected row - this is correct and no programming error.

I could not understand why the variable pin was given a value of -1. I tried

int pin;

[/code]


and there seems be be no difference to the system.





It's good programming practice to initialize variables if they are changed by if() conditions w/o a final "else" condition which guarantees that pin is set to a explicit value.

It protects my own code from unexpected conditions, but also the code of somebody else if he adds changes w/o understanding the complete routine.



You have to consider, that a simple "int pin;" without initialisation means, that the content of "pin" will be undefined.

It could have any number!

Maybe not today, but maybe tomorrow after you changed anything else in the application...





Another reason why I added this "-1" initialisation was, that I wanted to know if an unexpected condition happens at your side which I haven't considered in my formula.



Therefore I added:

[code]

  // following check ensures that we never continue with an unexpected/invalid pin number.

  // e.g. this could address a memory location outside the last_timestamp[] array!

  // print a warning message in this case for analysis purposes

  if( pin < 0 || pin >= KEYBOARD_NUM_PINS ) {

#if DEBUG_VERBOSE_LEVEL >= 1

    DEBUG_MSG("WARNING: row=0x%02x, column=0x%02x, pin_value=%d -> pin=%d NOT MAPPED!\n",

	      row, column, pin_value, pin);

#endif

    return;

  }

so that we would get a notification about the potential programming error.

You asked me to undertake an experiment with various degrees of touch to the keys. Here are the results , highly subjective of course!

Hard 126

Normal 100 - 105

Semi 80 - 90

soft 50-60

very soft 1

You could please specify the measured delay instead of the velocity value?

Because the velocity value is already a result of the formula which applies the delay value.

We want to replace this formula, therefore we need the input parameter.

Best Regards, Thorsten.

[TK.]

P.S.: I changed the scan loop again, so that RC is also pulsed after SRIO shift.

Approach:

- latch DIN values

- read DIN, shift next DOUT pattern

- latch DOUT values

- period

it is possible to omit the second pulse (as we did before), but first I would have to know if the current version is working correctly!

(as I cannot try it with real hardware, there is always the risk that I overlook something...)

Best Regards, Thorsten.

[robinfawell]

Dear Thorsten

Here are the Delays that I measured in mS.

Hard 4, Normal 10, Semi 40, Soft 65, Very Soft 200 - 300 even 400

The last measurement is very contrived. It is based on achieving a velocity of 1.

Latest program

There seem to be a problem with first 8 notes. ie Note A0 to E1. I show the A0 the others are similar.

[32731.500] row=0xff, column=0x00, pin_value=0 -> pin=0, timestamp=1055233 -> IGNORE

[32731.500] row=0x00, column=0x00, pin_value=0 -> pin=0, timestamp=1055233 -> IGNORE

[32731.500] row=0x01, column=0x00, pin_value=0 -> pin=1, timestamp=1055240 -> NOTE ON (delay=7); velocity=122

[32733.263] row=0x01, column=0x00, pin_value=1 -> pin=1, timestamp=1056996 -> IGNORE

[32733.281] row=0xff, column=0x00, pin_value=1 -> pin=0, timestamp=1057014 -> NOTE OFF

[32733.281] row=0x00, column=0x00, pin_value=1 -> pin=0, timestamp=1057014 -> NOTE OFF

Note that there are 3 0's and 3 1's

Extra Switches

I have modified the code and the table to allow for 16 switches in the empty part of the Table. I think that the system should have some extra switches that can be scanned. My requirement is deliberately basic. At present I can think of 10 essential ones at present. Please let me know if I should send the latest version. The table will make this clearer. I have not coded the new functions only made allowances.

Regards Robin

Connectin Diagram MIOS 32 Fast Scan with extra switches.doc

[TK.]

Hi Robin,

could you please try again with the version I released at 18:34 (CET)

(somehow I knew that you will face the error before I come back from a nice walk in the autumn sun ;))

Improved velocity support once pins are mapped correctly again.

Thereafter you can integrate your own code.

Best Regards, Thorsten.

[robinfawell]

Dear Thorsten

Same as last time. Is the latest version 1107?

Robin

[TK.]

yes - are you still doing any modifications (e.g. the "row-=1" workaround)?

Because row=0xff will never be passed by the SRIO handler, it should work with this change!

Best Regards, Thorsten.

[robinfawell]

Hi Thorsten

I think that I had two app.c in the folder. The latest has been tested with MIOS STUDIO and is AOK!.

See the result of pressing A0. I also played all the notes with the softsynth.

Future Work

I need to dismantle the keyboard to a add few diodes to my adaptor board. Maybe I should allow a couple of switches for enabling one or two velocity curves?

MIOS STUDIO.txt

[TK.]

Thanks!

I optimized the loop again, now we are counting from -1 to 15, and handle row==-1 as an "initial state".

This leads to better readable code.

Could you please try again? I hope that I haven't added another error.

Velocity curve: I entered your measuring results into a spreadsheet and assumed, that each "level" reduces the velocity value by -30 (just an assumption)

It turned out that the curve is pretty linear:

So, instead of using heavy mathematical stuff (like log() or exp()), it could be sufficient to change KEYBOARD_DELAY_FASTEST and KEYBOARD_DELAY_SLOWEST into variables, and to switch between two value sets with a switch.

Best Regards, Thorsten.

[robinfawell]

Dear Thorsten

No errors detected. All is fine.

So, instead of using heavy mathematical stuff (like log() or exp()), it could be sufficient to change KEYBOARD_DELAY_FASTEST and KEYBOARD_DELAY_SLOWEST into variables, and to switch between two value sets with a switch.

Do you mean using a pot or Thumbwheel switch to control both variables?

Additional Interconnection Table

It may be useful to include in the docs for this examples a table showing more clearly the connections between the LH and the RH Keyboard and the DOUT / DIN IC Pins. This would be for your present version. I have attached the diagram sse below.

It would also be helpful to others to include the 88 note Fatar circuit diagram in the example docs.

I hope I have the ability to complete the project on my own. Thank you once again for your time and trouble in developing the keyboard fast matrix velocity scan system. There are always queries on the midibox forum for keyboard scanning systems. Your solution using time measurement is elegant and probably unique.

Best Wishes

Robin

Keyboard to Shift Register Connections.doc

[TK.]

Hi Robin,

I finalized the application (at least for my side):

- velocity delay values now defined as variables, so that they could be changed during runtime

- DOUT now always latched after SRIO scan, because I've a better feeling when outputs are already stable before DINs will be latched some 100 microseconds later

- VERBOSE_LEVEL set to 1 for best performance (increase it to 2 or even 3 to get more debug messages)

Could you please confirm that the "final" version works properly at your side before adding your changes?

Do you mean using a pot or Thumbwheel switch to control both variables?

For finding the best matching values, the usage of a pot would be perfect.

Alternatively you could add a CC handler which receives delay values from an external MIDI controller (disadvantage: only values from 0..127 could be sent)

Or an "input terminal" could be implemented which allows to receive commands from MIOS Studio - but this wouldn't really be worth the effort, although you like geeky solutions!

It may be useful to include in the docs for this examples a table showing more clearly the connections between the LH and the RH Keyboard and the DOUT / DIN IC Pins. This would be for your present version. I have attached the diagram sse below.

It would also be helpful to others to include the 88 note Fatar circuit diagram in the example docs.

alright, done.

I hope I have the ability to complete the project on my own. Thank you once again for your time and trouble in developing the keyboard fast matrix velocity scan system. There are always queries on the midibox forum for keyboard scanning systems. Your solution using time measurement is elegant and probably unique.

Thanks! :)

Yes, I hope that other people feel inspired and provide customizations for their own systems.

Once you've finished your program, we could release it in a different folder as an additional example.

Btw.: I'm planning to rename this thread in some days, because the topic "SRIO, DOUT, DIN, 16 bit , 8bit Confusion" doesn't match anymore.

Best Regards, Thorsten.

[robinfawell]

Dear Thorsten

The latest code is functioning properly for Debug verbose levels 1 and 2. I'm not sure about level 3.

Verbose level 1 No debug messages. Midi On/Off

Verbose level 2 Debug messages as below. Midi On/Off

[4480.781] row=0x08, column=0x07, pin_value=0 -> pin=78, timestamp=10894 -> IGNORE

[4480.795] row=0x09, column=0x07, pin_value=0 -> pin=79, timestamp=10908 -> NOTE ON (delay=14); velocity=121

[4480.900] row=0x09, column=0x07, pin_value=1 -> pin=79, timestamp=11013 -> IGNORE

[4480.918] row=0x08, column=0x07, pin_value=1 -> pin=78, timestamp=11031 -> NOTE OFF

Verbose level 3 Debug messages as below. Midi On/ Off OK.

[3934.484] row=0x08, column=0x07, pin_value=0

[3934.484] row=0x08, column=0x07, pin_value=0 -> pin=78, timestamp=113654 -> IGNORE

[3934.499] row=0x09, column=0x07, pin_value=0

[3934.500] row=0x09, column=0x07, pin_value=0 -> pin=79, timestamp=113668 -> NOTE ON (delay=14); velocity=121

[3934.622] row=0x09, column=0x07, pin_value=1

[3934.623] row=0x09, column=0x07, pin_value=1 -> pin=79, timestamp=113791 -> IGNORE

[3934.640] row=0x08, column=0x07, pin_value=1

[3934.640] row=0x08, column=0x07, pin_value=1 -> pin=78, timestamp=113812 -> NOTE OFF

Note generation is working.

Thanks again!

Robin

[TK.]

Great, fine! :)

Verbose Level 3 generates an extra message for the case that the decoding routine filters out DIN events, so that they are not displayed.

It's mainly for people who want to re-use the code, but have to change the decoding.

Best Regards, Thorsten.

[TAG]

Would it be possible to modify this project to add either another DIN or DOUT in order to achieve a 24 X 16 scan matrix? I am working on a 192 key velocity sensitive keyboard (axis-64 clone).

Is it just a matter of changing these:

// scan 16 rows

#define MATRIX_NUM_ROWS 16

// maximum number of supported keys (rowsxcolumns = 16*16)

#define KEYBOARD_NUM_PINS (16*16)

Thanks,

Tim

[robinfawell]

There is a strong recommendation not to exceed the 16 bit Shift Register Max. See Tutorial 009.

There are two possibilities that occur.

Use line drivers between the core and the SR's. See Duggle's work.

Use 2 cores and split the matrix.

Regards Robin

[TAG]

I see that the limit is 16 shift registers on both input and output due to signal integrity issues, but what I am suggesting would only be 3 shift registers on one chain (either input or output) and 2 on the other. Well below the limit of 16 shift registers.