SDcard file access - multiple file handles and RAM

[Lee]

Hi all,

I'm still waiting for my hardware to arrive, in the mean time I thought I'd check out some software issues...

I'm planning on experimenting with playing samples back from an sdcard to the i2s dac... I know that more than likely the card reading won't be quick enough, but I also wanted to quickly change which file i'm reading by opening multiple file handles using the file module (plus the fatfs / sdcard / spi includes).

However, the file module is written to explicitly deny more than one file handle to be opened, the comments say:

//! For the whole application only single file handlers for read and write

//! operations are available. They are shared globally to save memory (because

//! each FatFs handler allocates more than 512 bytes to store the last read

//! sector)

//!

I was wondering if anyone's looked into the ramifications (excuse the pun!) of changing it to allow more than one file handle to be open, and how badly the memory got affected - before I attempt to do the same!

Many thanks all

Lee

[Hawkeye]

Hm... do you have a use example?

I mean, do you intend to read the full files... or do you want to read from many files incrementally? Would memory buffering be possible (and switching to the next file, when the buffer is full)?

You could always concatenate file data on an external platform and use seeks within a large file to avoid opening up multiple files... "Fake files" within a large file, so to say.

Also, it would be interesting to know how much time it really costs to close and open a file on sd memory media...

Greets!

Peter

Edit: Sorry, did not properly read... a sample-playback engine... I think, somebody already tried that on a Core32, but if I remember right, there were speed problems because of SD latency... maybe a forum search might help. Cheers!

Edited October 12, 2011 by Hawkeye

[TK.]

However, the file module is written to explicitly deny more than one file handle to be opened, the comments say:

//! For the whole application only single file handlers for read and write

//! operations are available. They are shared globally to save memory (because

//! each FatFs handler allocates more than 512 bytes to store the last read

//! sector)

//!

I was wondering if anyone's looked into the ramifications (excuse the pun!) of changing it to allow more than one file handle to be open, and how badly the memory got affected - before I attempt to do the same!

I'm sorry, this "wording" is a bit misleading, because it refers to FatFS file handlers.

But I implemented a method which allows to open multiple files with a very low memory footprint so that more files can be opened than with directly used FatFS handlers! :)

Instead of the FatFS handlers, use file_t (my own structure) instead, and never access FatFS functions directly.

See http://svnmios.midibox.org/filedetails.php?repname=svn.mios32&path=%2Ftrunk%2Fapps%2Fsequencers%2Fmidibox_seq_v4%2Fcore%2Fseq_file_b.c as an example

Note that there are some additional seq_file_* sources in this directory for various purposes which give you more usage examples how to read/parse/write binary and ASCII files.

I don't see a problem for your project, especially because reading the file will be faster than required for feeding the I2S based DAC.

Usually opening a file can take some time, because FatFS has to scan the directory structure with multiple block accesses.

But if you already open the samples in separate file_t handlers, switching between them will be fast and painless :)

Best Regards, Thorsten.

P.S.: I gave you access to the programmers lounge

[Lee]

Thanks all!

Yes, i'll be interested to see how low I can get the latency between reading from different files. Ideally I would have an architecture where a very large ram eg 256Mbit would buffer the samples, and then directly DMA out of that... However although possible it's not massively straightforward given a RAM of that size will be dynamic (I've had previous fun a long time ago with designing an FPGA SDRAM controller and don't really want to go there now)...

Anyway, to start with I was planning to open perhaps 8 sample files from the SD card, and within a few ms of midi input play or mute them (mixing to the i2s DMA sample buffer) - so it will need to keep resetting the file position pointer every time a sample is (re)triggered. How the SD card latency will fare here I'll be interested to see. If using FAT cooked etc is too slow, it's an option to perhaps just bit bang raw sample data from fixed positions on the card if that removes some latency, but I'll see once I can get my hardware rigged up to try!

TK - your link looks like a great help, thankyou. And cheers for such a cool platform to play around with!

Best, Lee