Polyphonic aftertouch sensors: FSR, Hall, capacitive or optical?

[Arjan]

Hi,

New member here with a question regarding aftertouch. I've been thinking about ways to add polyphonic aftertouch to an existing keyboard that only has channel aftertouch (or no aftertouch at all). Currently I'm looking into what type of sensors I could use and a search here turned up only a few hits which suggested using FSR's (Force Sensitive Resistors). I have found a source for FSR's that may be suitable but they are still EUR 4,50 which is a bit much imo if you need 76 of them.

I was wondering if anyone has ever considered using a Hall effect sensor instead? I did notice in one thread that these have been used to MIDIfy organs but it appears in that case they were used as switches only. However I think a Hall sensor could be used to very accurately measure displacement, so how about measuring the 2 or 3 mm of travel that you get after a key hits the keybed?

I haven't found reliable price information yet but it appears that perhaps something like a TLE4990 (Infineon) might fit the bill. If the physical measuring range can be made broad enough (say 6mm) while maintaining sufficient resolution and accuracy (I guess about 50 discrete levels per mm would be sufficient) that would allow for a relatively loose construction with an electronic only calibration.

I've also been thinking about some optical system using photo diodes/transistors or photo resistors but that seems more convoluted and difficult to construct. For one thing it would require active components on two sides. With the Hall effect sensor one side would consist of just a tiny permanent magnet which could be glued to the bottom of the keys.

Another approach might be purely capactive but that seems more difficult too. Plus the Hall effect would suffer very little mechanical stress (basically just the keybed which may deteriorate over time requiring either replacement of the keybed felt or rubber strip or, if it is still elastic enough, re-calibration of the sensor readings.

Any thoughts?

Arjan

[stryd_one]

Welcome aboard arjan :)

Interesting idea, but not too simple... The short answer is yes this could probably be done... With a fair deal of effort.

Things that require consideration:

You're probably going to need per-key calibration unless your sensors are really accurate

You'll need to write a custom multiplexer driver that can address all of those keys, and it would probably make sense to optimise it to read from the keys which have reached a voltage you use as a threshold (as in, when the key is held in, track it's movement). That said, the MIOS AIN mux driver already has a similar optimisation (reads from AINs which have changed) and core32 supports 128 AINs. Perhaps that would be a platform worth waiting for (it'll be ready when it's ready, as they say...)

Once you've got it working mechanically and electronically, then it's a matter of writing an app to deal with the voltages and convert them to the combined on/off and aftertouch messages.

I hope you've got your experimenter's hat on :)

[/tilted/]

perhaps even simpler, you could take which keys are held direct from the MIDI stream...

[Arjan]

Welcome aboard arjan :)

Thanks :)

You're probably going to need per-key calibration unless your sensors are really accurate

Most definitely. That's why I'm hoping to find a sensor that will be accurate enough over a broad range of say 6 mm so hat the actual measuring range of 3mm is guaranteed to fall within the total sensor range to allow high tolerance in physical construction and completely electronic (per key) calibration.

You'll need to write a custom multiplexer driver that can address all of those keys, and it would probably make sense to optimise it to read from the keys which have reached a voltage you use as a threshold (as in, when the key is held in, track it's movement).

Actually I was planning on parsing the normal MIDI output stream of the keyboard for note on/off messages to keep track of which ones are pressed and only scan those for pressure.

That said, the MIOS AIN mux driver already has a similar optimisation (reads from AINs which have changed) and core32 supports 128 AINs. Perhaps that would be a platform worth waiting for (it'll be ready when it's ready, as they say...)

That would be nice and simple. But on the other hand if I don't multiplex I would have way more circuit lines going to my sensors than if I do multiplex. I haven't really looked into this yet, first I need to find a suitable sensor.

Once you've got it working mechanically and electronically, then it's a matter of writing an app to deal with the voltages and convert them to the combined on/off and aftertouch messages.

I intend to do initial development and testing using a PC. I have alread written a small app that can simulate polyphonic aftertouch. The way it currently works is that if you hit a key again (while holding down the sustain pedal) the note on message is tranlated to a poly aftertouch message and sent back to the keyboard instead of a note on (note off also gets suppressed while the pedal is down).

It works but it doesn't seem to make for intuitive playing. I'm going to do another experiment where each keypress will increase the aftertouch value, perhaps depending on the time between keypresses (shorter is larger increase) and a certain 'release' curve to get back to zero.

I hope you've got your experimenter's hat on :)

I'm not sure yet, but I do have a masters degree in electrical engineering and plenty of programming experience so I can do this if I want to. Just have to figure out if it's worth my time and trouble but it would be fun to come up with an affordable quality solution to retrofit poly aftertouch to existing keyboards.

I may just have to order some Hall sensors to see what sort of readings I can get from them but I was hoping that someone could advise as to the feasibility of using them, perhaps even suggest a specific model.

[Arjan]

perhaps even simpler, you could take which keys are held direct from the MIDI stream...

Exactly what I planned to do. Your response came just seconds before I posted my previous reply.

[Futureman]

Hey there, Cool idea.

I'm not sure calibration would be thaaaaat important... It's aftertouch after all.. Say, you want to lift out a note from a chord you held down (Via filter or volume).. is it really that important that it's aftertouch sensitivity has exactly the same response as other notes.. if not, then you could sure save yourself a bit of work.. assuming you have a reliable way of placing the magnets (And sensors) in the same position?

Even velocity sensitivity is not hugely important to me (Well calibration wise).. have you ever used a Rhodes or Wurlitzer electric piano?

FYI the Korg Sigma uses a hall effect device for it's aftertouch, but it's just boring aftertouch, not poly... The SCI Prophet 8 used optical sensors for it's Poly aftertouch & velocity I believe.. not sure what the Yamaha CS-80 used.

My 2 cents anyway..

[Arjan]

I'm not sure calibration would be thaaaaat important... It's aftertouch after all.. Say, you want to lift out a note from a chord you held down (Via filter or volume).. is it really that important that it's aftertouch sensitivity has exactly the same response as other notes..

I don't think it has to be super-accurate but if you're playing a chord without applying any intentional pressure I'm sure you'd want your notes to play equally loud (approximately). So the starting point of the response curve is fairly important I think unless you are happy with a large threshold value.

if not, then you could sure save yourself a bit of work.. assuming you have a reliable way of placing the magnets (And sensors) in the same position?

I think calibrating through software to make up for tolerances in the mechanical construction will be a lot easier than putting heavy precision requirements on the mechanics. The travel range is only about 3mm so without calibration the mechanical accuracy would have to be something like 3/127 of a mm at least. Plus there is sure to be spread between sensors' response curves and also, in the case of a Hall effect system, in the strengt of the individual magnets. I think calibration will be mandatory.

FYI the Korg Sigma uses a hall effect device for it's aftertouch, but it's just boring aftertouch, not poly... The SCI Prophet 8 used optical sensors for it's Poly aftertouch & velocity I believe.. not sure what the Yamaha CS-80 used.

That's interesting, didn't expect the optical approach to have actually been used in a commercial product. And nice to know that at least one synth has used Hall effect for aftertouch.

My 2 cents anyway..

Appreciated, thanks.

[Futureman]

The travel range is only about 3mm so without calibration the mechanical accuracy would have to be something like 3/127 of a mm at least.

Lol.. I don't think you would need to impose a 3/127 of a mm accuracy... seriously, whats the difference between aftertouch value of 1, compared to 7?

But, yea, it seems like software calibration probably would be in order..

Small things to consider might be the fact that magnets from adjacent keys would effect the sensors not under them... but then again, how often do you play C and C# and D at the same time while using the aftertouch on C# and not the others? Unless you like the psycho violin theme riff lol.

Also, maybe watch out for the inverse square law of the magnetic field...

Keep us posted.

Regards

Mike

[Arjan]

Lol.. I don't think you would need to impose a 3/127 of a mm accuracy... seriously, whats the difference between aftertouch value of 1, compared to 7?

Yeah, I agree that 3/127mm may be overkill. But to me even just <1mm tolerance does not seem that easy to accomplish over the full length of the keyboard. Unless you mount both the sensor and the keybed on a single PCB/carrier, then the length of the keyboard becomes largely irrelevant.

But what do I know, I'm an EE, not a mechanical enigneer. :D If all you've got in your toolbox is a hammer, a screw will look like a nail. My toolbox contains microcontrollers, software and other eletronics so that's where I look for solutions :D

[stryd_one]

Exactly what I planned to do. Your response came just seconds before I posted my previous reply.

Err, maybe I'm missing something but that sounds kinda backwards...ie, won't work. The mux (and you DO need it, there are only so many AINs on the core, you can't run 76 keys worth of analog inputs direct to the core) will be running at a driver level and only once it has finished reading in all of the voltages for all of the keys will your application be notified of that read. So by the time you send MIDI, it's too late to change the mux behaviour.

Besides, Why would you convert AIN X to Note Y and then use Note Y to adjust the behaviour of the mux and how it reads AIN X (and all the others)? why not just use the original input of AIN X ?

[/tilted/]

I'm suggesting that as an optimisation, you could use the key on data to determine which AINs to read.

In terms of speed, it is surely not vital that you send aftertouch along with key velocity. It is after all, called AFTER touch...?

[Arjan]

Err, maybe I'm missing something but that sounds kinda backwards...ie, won't work. The mux (and you DO need it, there are only so many AINs on the core, you can't run 76 keys worth of analog inputs direct to the core) will be running at a driver level and only once it has finished reading in all of the voltages for all of the keys will your application be notified of that read. So by the time you send MIDI, it's too late to change the mux behaviour.

I'm not yet familiar with the details and limitations of MidiBOX/MIOS but in general the principle seems sound. As soon as you receive a note on you add it to the list of notes to monitor for pressure change which you will need to do many times a second anyway (as often as possible). Reducing the list of notes to monitor can only increase the number of times you can update the pressure information for all notes that are currently pressed, right?

If you don't use the MIDI stream from the keyboard you'd have to check all keys for pressure info. I suppose you could increase the monitoring frequency of certain keys once you have detected that they have travelled down into the aftertouch range. Which method give higher performance would depend on the latency between actual key down and receiving/processing the MIDI note on event and the maximum monitoring frequency that you can obtain if you are monitor all keys for pressure info.

Besides, Why would you convert AIN X to Note Y and then use Note Y to adjust the behaviour of the mux and how it reads AIN X (and all the others)? why not just use the original input of AIN X ?

I'm not sure I follow what you are trying to say but that could be due to my unfamiliarity with MidiBOX/MIOS.

[stryd_one]

I see where you guys are coming from now.. I'll try to explain in greater depth...

What happens at the moment, is that each 100us, a timer triggers an interrupt, which causes the AIN handler to be run. This reads in 8 analog inputs (each of them a pin on the PIC chip) simultaneously... by using the multiplexers, this 8 inputs can be increased to 8x8=64 inputs, read 8 pins at a time. This reads pin 0 on all multiplexers, then pin 1..2..3.....7.

If the values read have changed, then the AIN_NotifyChange hook in the application is called. The arguments passed to this function, tell us which pin has changed, and what value it is now. This is where you can deal with the new input, and process it in some way - in this case, comparing the value to a threshold, and sending a note-on message according to the pin number that has registered a change.

If you wait until this point, you can continue to read the relevent data (the value of the keys that are held in, that you want to track for aftertouch) from the PIC's memory at a higher rate. This is a good idea, and I think this is what you're suggesting.

The issue with this, is that the PIC and the OS, will still be having to read from the muxed AINs at the same rate as always, so you'll just be reading the result of the normal analog read from RAM more often, which won't actually get you any increase in resolution. If you optimise the mux driver, then you can have the relevant analog inputs actually scanned more often, which will give you better resolution for your aftertouch.

Normally this would not really be so necessary, but as you will be writing a custom multiplexer to use more than 64 keys, it makes sense to do this, to get the best performance.

I didn't want to confuse the above topic too much, but as I mentioned, the MIOS mux driver does already have some very similar optimisation in place, to what I have proposed here. It is optimised for human hands - it reads more frequently from the last two changed values (left hand, right hand!). This optimisation in itself may be enough.... Perhaps you'll want to monitor the last 10 (fingers) more heavily... I'm no pianist  :-[

FYI, the difference in core32 when it comes out, assuming there are no changes (which is unlikely, but no promise, it is in prototyping after all) is that it has 12 analog inputs on the chip, so the 8 into 1 multiplexers can read 8x12 = 96 inputs. The analog inputs are also read using the DMA controller, so it happens in the background with minimal interruption to your app. it just starts the read, goes away and continues with your app, and when the read is done by the DMA controller automagicallyin hte background while your app runs, it calls a function to let you know. arjan you are probably familiar with callback schemes already so you know what I mean there. Guys who are used to MIOS8 will be impressed by this new feature :) As far as I am aware (TBH I haven't checked yet!) the MIOS32 AIN driver has the same 'last two pins' optimisation as MIOS8 does, so if you're considering holding off until mios32, perhaps that will be sufficient... perhaps not...To use one example you mentioned, if you are slowly releasing just one key while holding the others, perhaps this will be spot on. It depends how stable your hold on the other keys is :)

Sorry the info is not highly specific, as I say I'm not much chop on keys (I'm a strings and skins guy myself) but I hope the tech info is helpful!

[Arjan]

I see where you guys are coming from now.. I'll try to explain in greater depth...

Thanks for clarifying, thinks are starting to make more sense as I get a better understanding of how MIOS works. I didn't even know that there was built-in multiplexing support but I understand modifying the driver for that is an option.

I'm still in the very early stages of this project and I expect I will need quite some time to settle on a sensor and the mechanical construction. For me I expect construction will be the biggest challenge so I will probably have time to wait for this core 32 thing. Like I said I intend to develop this on a PC initially. I will probably not implement the multiplexing on the PC but I do have a USB interface card somewhere which also has some (4 or 8 ) ADC converters that I could use to test the sensors and the actual practical use with just a few keys (if I can remember where I put it).

Sorry the info is not highly specific, as I say I'm not much chop on keys (I'm a strings and skins guy myself) but I hope the tech info is helpful!

It is and very much so, thanks! I do indeed know what callbacks are, I live and breathe callbacks in Windows programming ;) The DMA controller will be a great improvement I'm sure.

[dbolton]

An old post, but very interesting for me, since I am considering such a project as well. Any progress over the last years?