Yes, maybe implement the option to hold a few very small (looped) samples in memory (those single-cycle waveforms) or other wavetable-ish stuff.
Me still haz some code of my old (1994) dos based mod player somewhere (which does sample stretching to obtain the played note pitch - good old amiga times :-)),
but then... this would be quite another project... Lee decides :)
Edit: on a second thought, SD storage is so big and cheap, one could totally create full note spectrum sample files of "unlimited length" offline, that means, if you have a nice single-cycle waveform, run a script that "renders" a samplebank for the SD player, as it is... not much work to do, and it runs on todays sd card sample player software :) and does not require further ram or processing power.
Greets,
Peter