Serial SPI Flash Memory

[This N°9]

hi,

I was looking for storage which could be used with MIDI-box. My target is to build a non-step sequencer (clip based, like ableton live) and need non-volatile storage that is fast enough to read/write MIDI-streams realtime.

First I planed to use banksticks and use RC4/RC5 pins to multiplex, so I could access 4x8 banksticks, I would have 2 MBytes of storage available (32x64KBytes). My plan is to write a very simple filesystem, that allows me to handle clips with variable size.

About speed: The 24LC512 datasheet says, that a write cycle (64bytes or one byte) takes max 5ms. This is quite much but would be fast enough if I reduce the sequencers' maximal resolution to 1/128 rather than 1/384 (MIDI clock). With 120 BPM one "step" would be ca. 30ms, this is enough to play back and eventually write 64bytes of data to the bankstick.

When I was looking for alternatives for the bankstick concept, I found this part:

http://www.spansion.com/products/S25FL016A.html

It looks quite interesting, it's fast (256 bytes in 1.5ms), even 64Mbit/128Mbit is available for a good price:

http://at.farnell.com/jsp/search/browse.jsp;jsessionid=R4ZQVHLMSR2AOCQLCISZKBQ?N=0&Ntk=gensearch_002&Ntt=S25FL&Ntx=&_requestid=233886

maybe this is interesting for somebody else, I will check out a bit more. This would save me from having 32 IC's for my storage, and save some time for the writes.

[ilmenator]

The answer is: SD card module, look here.

[TK.]

And especially here:

SD Card Driver

Demo Applications

Best Regards, Thorsten.

[ultra]

more about the sequencer? :)

[stryd_one]

clip based, like ableton live

It was the timestretching, while a clip is already playing, that always messed with me on that concept. Some of our newer toys should do the trick nicely ;) Good luck man, keep us posted!

[This N°9]

more about the sequencer?

why I want to build a thing like this: I was always looking for a ableton-live-style sequencer, but this simply doesnt exist on the market. I dont want to use a personal computer for music, I sit in front of a monitor all day, and I want to have a more dedicated device with the proper buttons etc.

I shopped a MV-8000 by roland, which is really a very powerfull and nice device for composing, a dream machine, you can even put it in sampler mode, if you do so, you can access all parts(instruments) on it in parallel by midi. Then I have a paia modular system for bass sounds etc. and a muse research receptor for VST plugins.

My way of composing usually goes like this: I have some idea for harmonies or a bassline, I want to play it on the masterkeyboard and record it as quick as possible. I can do this with the MV, but I first have to  setup a new song, select the channel for the track etc. This takes too long. Besides the MV's live jamming features are limited, it's more a composing tool than a live tool. So I need a machine that uses exactly the principle that ableton live does, you can trigger clips in any combination you want.

Here the target features of the machine:

- 16 vertical "strips" each one targeting a midi-channel.

- each strip has 8 or more buttons for clips. eventually I will implement a paging, so you can have more clips.

- each clip has a length, can be one-shot or loop

- for each strip, one clip can play at one time

- one clip can be recorded at one time, by simply hold the record button and push the clip. record modes are overdub/replace, I would like to have the replace mode only replace until you played the last note, so in fact the last "take" will last.

- max. resolution will be 1/128, this gives me 30ms@120BPM / 15ms@240BPM to stream data from/to the storage in realtime, which would be even enough with a bankstick as storage. With a flash, this would be even better.

- one you have some clips playing, you can store this as a "scene" or whatever it's called. Scenes work like clips, you can push the scene button, and the clips start to play. Then you can play arround again.

- a song is  just a set of clips and scenes.

It was the timestretching, while a clip is already playing, that always messed

should be no problem. Between the single MIDI events I will place "spacer" events, they just hold the number of "ticks" to wait until the next event. I align this to a global counter, that is driven by a timer or external MIDI-clock. I'll have to handle counter overflow somehow. If you change playback speed, the only thing that happens is the counter increase speed.

The first challenge is just to build the app for a single clip. Then "clone" the clips and check if the core can handle this.

[This N°9]

..a future project could be a sampler based on a pico ITX board and a linux with a dedicated app. The device could be used independently just as a sampler or with the sequencers communication over MIDI with dedicated commands, so also audio clips could be recorded/played back by the seqeuncer. A challenge would be to implement the time-streting algorithms, but I think there will be some ready-to-use code examples arround I hope.

[This N°9]

@TK

very nice, the SD card stuff! I think this perfectly servers my needs.

[stryd_one]

If you change playback speed, the only thing that happens is the counter increase speed.

Yeh but increasing speed on individual clips means a counter for every clip. No problem on core32 :)

You know you pretty much just described the vX, just not modular? ;) The only difference is that you're not talking in steps... I would imagine this could be done as a vX module :D Problem is that if you don't need the modularity then you'll lose some processing power ... no good.

Edit: but I'll still wanna modulize it ahahah

[This N°9]

Yeh but increasing speed on individual clips means a counter for every clip. No problem on core32

I wont do that. Just one counter for all clips. Increasing speed for a single clip would mean to modifiy all "pauses"

accordingly, e.g. double/half them.

You know you pretty much just described the vX

what's the vX?

[stryd_one]

I wont do that. Just one counter for all clips. Increasing speed for a single clip would mean to modifiy all "pauses"

accordingly, e.g. double/half them.

Gotta watch out there cause you can make it drift. Like, think if you slowed the clip down and then sped up... If your speeds are locked to 2^n then you'll be right though...

what's the vX?

Stands for 'vapour Xtreme' heheh... sorry it's a joke we make cause i've been trying to finish it for like 6 years ;) I have $#*% luck. It's a modular polyrhythmic loop sequencer. I keep getting asked what it is since I pulled down some old doco on the wiki, I've kinda been procrastinating cause it makes no sense without pretty pictures and I'm bad at that ;) I'll go write something new and post it elsewhere...  Better not to hijack anyway!

[This N°9]

Gotta watch out there cause you can make it drift. Like, think if you slowed the clip down and then sped up... If your speeds are locked to 2^n then you'll be right though...

The seqencer's target is not so much to play arround with the speed of a single clip. maybe you could double/half the speed or so. even if there is a pause of say 19 ticks, and I can't half this, there is still the fixed length of the clip itself, that ensures that it will be triggered at the right point in time again. But this is a good point, maybe I would have to keep track of the clips last starting point.

About memory consuption: I'll limit the clips playing back at once to 16, I think for my needs this is enough (1 bassline, 1 harmony part, 1 melody part, drum part, effects part, ....., ...., ....). So I'll have a struct for each playing clip, maybe

- current pointer in file

- next event playback time

- last clip start time

if each of these values is a int, I need 16x6 = 96 bytes.

for recording (only one clip at one time) I need a buffer of  256 bytes

as a global counter, a int (max pause length is 2048 quarter notes then, taking overflow into account).

another challenge will be the playback streaming. without any buffer, in worst case I have to read 16 blocks, each takes 3ms to read, so this ends up 48ms just for read delays. this would be too much if my ticks are ca. 30ms for 120BPM. So maybe a playback buffer would be usefull, say 128bytes and a smart buffering concept.

this would end up in ca. 500 bytes memory needed for the app. I hope this fits, taking the memory consumption of the FAT driver and MIOS in account... If not, I may need 2 or more cores, one just as a "file-server" or so...

[stryd_one]

500 bytes is realistic, no sweat. I think you're right about the need for buffering though.

[This N°9]

I saw that with FAT I can't append data to files. Thats bad, maybe I will build my own little FS with just numbered files, that would save me quite some memory for buffering etc. The concept for the FS is already worked out (originally for Banksticks), I just have to alter the blocksize. No more chipselect, this is nice also.

I just ordered some SD cards, slots for them, and also some serial flash IC's that I mentioned above, I think they should work with the SD driver also? they support clock frequency up to 50MHz, and use SPI as protocol. I will check what I will use in the end, maybe I'll use the IC's, when I use my own FS, then I have no advantage to plug it in and out. If the memory is dead, I have to open the box and change the chip, but I  think it will last some years.. I think a sysex-backup will do it also.

UPATE: the Spansions IC's use a different command set than SD-cards.

[TK.]

I saw that with FAT I can't append data to files

But you could prepare placeholder files (e.g. bank1.bin, bank2.bin, ... bank1000.bin) which are large enough to hold as much data you need. See also Ilmenator's project.

maybe I will build my own little FS with just numbered files,

Advantage of FAT: you can easily copy files between your PC and MIDIbox w/o time consuming SysEx transfers

Disadvantage: searching files in a large directory increases the latency if cluster references are not buffered in RAM. And once the number of sectors is higher than a single cluster, you will notice some additional latency due to FAT accesses.

Advantage of your own FS: you can optimize it for your needs (e.g. direct sector addressing, no allocation table)

Disadvantages: reverse advantages of FAT

However, for your project I guess that using an own FS is the better idea due to low latency requirements and RAM limitations of the PIC

I will check what I will use in the end, maybe I'll use the IC's

There is no advantage by using these serial flash chips, since the PIC slows down the data rate, not the memory (SD Card is already accessed as fast as possible)

UPATE: the Spansions IC's use a different command set than SD-cards.

wasted money, wasted development time...

Best Regards, Thorsten.

[TK.]

this would end up in ca. 500 bytes memory needed for the app. I hope this fits, taking the memory consumption of the FAT driver and MIOS in account... If not, I may need 2 or more cores, one just as a "file-server" or so...

or a single STM32F103RE for EUR 12? ;)

Note that a lot of code (e.g. timestamped MIDI handler which allows to pregenerate streams) already exist, since MBSEQ V4 - but also vX32 - are working very similar... but it seems that you are just searching for a challenge ;)

Best Regards, Thorsten.

[This N°9]

or a single STM32F103RE for EUR 12?

hm.. maybe this would be a good way to get into the whole MIOS32 story.. I guess there are bulk orders running? My last impression when I stook my nose into the MIOS32/MBHP32 threads was, that stuff is almost ready to takeoff?

but it seems that you are just searching for a challenge

this is also true... and I have some already built core modules left. When I get stuff running on a slower smaller core, this has no disadvantage. The speed of the MIDI-transmission itself stays the same for both..and if I run into the wall, it's no big thing to port the application for the new core module (which will be a good idea anyway).

wasted money, wasted development time...

hmm.. maybe. I think I'll check both variants anyway. Of course theres no speed advantage in using the IC's, and I'm not sure if they have life extending stuff built in, but maybe someday somebody for any curious reason prefers a IC. I'll stick my nose a bit inside your SD card code, learn some ASM, I think it would not be a big thing to port the code for the Spansion.

[This N°9]

just to continue the story: I studied a lot of specs and asm code (sd card module, spansion spec, sd card spec ..), here some hints & questions:

the spansion IC's are not made for this kind of usage, 1st they have no kind of controller mechanism for erease/write/sector assign etcetc. even doing this on software side is hardly possible, because one sector is 256Kbits, and has to be ereased before re-programming. The IC's could only be usefull for bigger data ammount backup/restore. But the protocoll is very flexible, so byte-read will automatically increase the address pointer to the next byte, as long a clock is received, the IC will output bytes.

Anyway it's not usefull for my seq project. The problem with the SDcard again is, that only some cards seem to support block reads of less than 512bytes. The problem is that this is quite a lot of data for the poor PIC, and if I ignore some of it, it will be just wasted CPU time. 3mS*16 (worst case read time for 16 streams) is really a lot of time. Maybe I'll just use a bankstick array of 32 sticks (->2MBytes) and use the SDcard as backup device to share with the PC. The read/write on the BS is fast enough, and 64 bytes is still ok if I send some bytes immediately and buffer some more, so redundant read access will be not so dramatically high.

@TK

I studied your SD-card asm code (and learned some asm by the way..), now I'm a bit confused about the usage of the unit "mS": in the code you have a helper macro called "SDCARD_1mS_Delay". I assume that here it means "micro" not "milli" (?) The second thing that makes my brain smoke is the line "bra $ + 2". I understand that the idea is to perform 2 cycles and go to the next command, but in the PIC18F452 datasheet I see this definition for the "BRA" command:

BRA nd ;    PC+2+2*nd->PC"

If this is the correct reference I'm reading, "bra $+2" would branch to $+2+($+2)*2 rather than to the next instruction, or will the use of $ cause some special handling by the assembler?

thanks for your inputs

[stryd_one]

As for the BRA command, the PC is measured in bytes, but the instructions are usually 2 bytes each :)

(This is the same reason that the PIC16 port is so named, despite being an 8-bit chip)

[This N°9]

As for the BRA command, the PC is measured in bytes, but the instructions are usually 2 bytes each

yes, I'am aware of that, but the PIC spec (PIC18FXX2->Enhanced 16-Bit Core Instruction Set (75 instruction set)) gives the following definition of the BRA instruction (see screenshot):

This is from another spec version:

Mnemonic: BRA nd

Description: Unconditional relative branch

Function : PC+2+2*nd -> PC

It's a bit confusing, I suppose the PC+2+2*nd will only apply if n is < 1024.

[stryd_one]

I fail to see the cause of the confusion?

[This N°9]

I fail to see the cause of the confusion?

seems to.. or I missunderstand something:

PC+2+2*nd -> PC

nd is the param for BRA. The spec says that the above formula will be performed to set the new PC. But it seems that it is also possible to just branch to a absolute address. But the BRA is described as "relative branch". This confuses me.

[stryd_one]

PC+2+2*nd->PC"

PC+2+2*nd -> PC...

PC+2+2*nd

PC+2+2*nd -> PC

LOL. I read it the first time dude! (OK I read it in the DS, but yeh... I read it!) I think we have language barrier... I'll leave it to TK :)

[This N°9]

I give it one more try.. :

BRA $+2 ;I want to branch to the next instruction

assume $ is 0x222222, $ == PC

new PC = 0x222222+2+2*0x222222

the result of the above formula will for sure *not* equal to 0x222224 !

?

[TK.]

Just check the opcodes in the listing (project.lst):

SDCARD_1uS_Delay
003ae8   d000     bra   0x3aea                  bra     $+2
003aea   d000     bra   0x3aec                  bra     $+2
003aec   d000     bra   0x3aee                  bra     $+2
003aee   d000     bra   0x3af0                  bra     $+2
003af0   d000     bra   0x3af2                  bra     $+2
[/code]
Offset is 0 -> PC will jump to +2 location
If the assembler would handle labels the way you explained, you would have to write
"rcall SDCARD_1uS_Delay-2" instead of "rcall SDCARD_1uS_Delay"
Btw.: I changed the function name.
Oh - and before you are speculating about delays even more: in fact, calling the 1uS delay function will consume more than 10 cycles, since the call itself + the return instruction will cost +3 cycles (so far I remember)
Hope that you don't mind, that I don't change this - it doesn't hurt to clock the SD card a bit slower after power on
Best Regards, Thorsten.