Sauraen

I tested my pvPortRealloc, and after fixing the bugs and testing what I believe to be one instance of every case of resizing a block, I committed the implementation. It changed heap_4.c, portable.h (to provide a definition), and freertos_heap.cpp (to make realloc point to this function). I documented these changes as you requested in README_1st.txt. It should not have any effect on any existing projects, as (evidently) none of them use realloc, and I didn't change any other code in heap_4.c (besides adding an additional printed parameter to your vPortMallocDebugInfo function). MIDIbox SEQ V4 compiles and appears to run fine on this new version. Feel free to test my implementation as you see fit.

I wrote a pvPortRealloc function for heap_4.c and linked it up the way the other OS memory management functions are, and now the synth compiles and runs fine. But it's barely using the realloc functionality as is, so I'm going to write a little interactive test program for the synth and see if I can stress-test it and find any bugs. Once I do I'll commit it and you can give it a try. Of course it won't affect any current projects, because none of them use realloc, but of course it is important to have correct for the future.

Found the problem: it's realloc. I created a copy of the C template and it compiled fine. I added the lines u8* a = malloc(16); a = realloc(a, 32); to app.c in APP_Init() and it fails with the "undefined reference to '_sbrk'" linker error. I use realloc a couple times in the VGM module, and it's also used by my vgm_heap2 implementation of the secondary heap in that extra 64k of RAM, in so far as that heap provides its own realloc function to go with its own malloc and free, and it calls the OS's realloc function if the pointer you passed was in the main heap rather than the secondary heap. While I could get rid of these all without too much trouble, I'm not sure why a major embedded platform like MIDIbox would choose a memory management scheme (heap_4) which does not support realloc, especially if it used to support this (with umm_heap). I could write an extension to heap_4 which provides realloc--it's not a terribly complex system (it's simpler than umm_malloc)--but for the sake of quality control it might be better to use a real-world-proven solution (and I don't exactly know how to tell the linker that whatever "pvPortRealloc" I write should be called whenever "realloc" is called).

Two things to point out which might lead to solutions: When I compile anything (even midibox_seq_v4, which succeeds), I get a new warning along the way "warning: stack usage computation not supported for this target". Thought I'd mention it since it's memory-related, but that's stack and not heap so it probably isn't relevant. The other thing I could think of is that MIDIbox Quad Genesis uses some of the STM32F4's hardware timers, so MIOS32_DONT_USE_TIMER is set in mios32_config.h. In case one of these timers is being used by the OS, maybe that's what's going on. (It's Timers 2, 3, and 5, and of course it worked fine with the old FreeRTOS, so unless it uses new timers there should be no issue.) Worth mentioning, commenting out MIOS32_DONT_USE_TIMER results in a "multiple definition of TIM3_IRQHandler" as expected, but it also does not get rid of the "undefined reference to '_sbrk'" error. Disabling my entire secondary heap (in the 64 kB of non-DMA RAM) and all calls from MIDIbox Quad Genesis / VGM Module code to the OS's memory management stuff also did not solve the problem. Just checked again that SVN is up-to-date, and all my stuff is committed, so you should be able to go into apps/synthesizers/midibox_quad_genesis and try making it if you want to see.

I finally got around to converting my code for MIDIbox Quad Genesis / VGM module to use the up-to-date heap_4, which was not hard because it already has a system for monitoring the memory usage. However, now when I tried to build (after "make clean" of course) I'm getting the linker error "undefined reference to '_sbrk'", which the internet says has to do with some memory management libraries not having been compiled. My SVN is all synced and up-to-date, and midibox_seq_v4 compiles fine. I tried editing my Makefile to look more like the midibox_seq_v4 one, and there wasn't much to change (I of course left out modules which shouldn't matter), though I tried adding in the "freertos_utils" module but that didn't help. I also tried getting rid of the part of my synth's code which directly interfaces with the heap stuff, and that didn't fix it either. Don't know what else to do--my MIDIbox Quad Genesis project is just a src folder and a Makefile, so there's not much place for there to be something wrong.

Unfortunately I've been busy with grad school, so I haven't had much time to work on the software recently. I did do a bit during the fall though--editing VGM files from the interface, by command or state, is beginning to work. As far as your request, with the current MBQG code, muting a voice doesn't release it into the pool of unallocated voices, it just sends a key-off to the sound chip and proceeds to ignore future key-on messages. However, the voice does NOT ignore other parameter changes, since when you unmute the voice, you want it to continue playing from where it is (in case parameters change), not continue playing using the old parameters. It would be relatively easy to change the code to have it release the voice and re-allocate it later, but then it would usually sound wrong when un-muted. It might be possible to add a feature where, for a muted voice, you can play notes on it but it continues to obey all other parameter changes in the VGM file. This would solve your problem, though you would have to accept that parameters might change while you're playing. And also, there would have to be a menu option or something which would allow you to select what MIDI channel to use to play that voice, and that might not be convenient to set up while playing. However, if you are using MBQG with two or more Genesises, you can simply mute a voice and play the same sound on some other voice (usually on the other sound chip). The operation of extracting the state of a voice while playing and assigning this to another MIDI channel is a core feature of this synth--it's not quite in place, but it's definitely planned, in fact there's a dedicated Capture button for this. You would select the voice, press Capture, and then press the button for the MIDI channel you wanted to assign that sound to, and it would do all the work of creating the three VGM files for that channel and setting everything up, so you could immediately play that sound on that MIDI channel. Are you planning to build a MBQG with the full interface? I have front panel boards which I will be selling at some point--if you're interested we can work something out.

Thanks for letting me know. I haven't been actively working on this project recently, but when I get back into it I will see what changes I need to make. I may just end up staying with the modified umm_malloc.

You want to make a OPL3 or OPN2 synth? If you're talking about the latter, no need to write code or change the synth engine! You have three options: MIDIbox Genesis Tracker, which is just a STM32 core and the Genesis board, and allows you to control all the sound chip registers via MIDI commands. No synth engine. Easy project. Beta code available, you'll be the guinea pig. :) MIDIbox VGM Player, which is the core, the Genesis board, and a very small control surface (similar to Standard Control Surface but connected via DIN J8/9 not digital J10). Lets you play VGM files on the Genesis board(s) from an SD card. Release code already available (though not up-to-date, I will update it if you want to build this.) MIDIbox Quad Genesis, the synth with the large front panel and hundreds of LEDs. Polyphonic multitimbral synthesizer with a synth engine focused on "sampling" VGM files. Software in progress, see the videos linked on that thread for more info.

New video, though I didn't take the time to record the sound the right way. It's showing the VU meter feature, which uses undocumented test modes of the OPN2 to read back operator states while the VGM file is playing. Also show off the fabled SSG-EG mode which turns the EG into a glitchy LFO, and proves that the VU meters are not being emulated.

New video! Goes over the way instrument programs are structured and how everything is made out of VGM files.

It's committed, have fun! I made brief notes of the changes in the file you requested, plus there's a decent amount of comments on the changes in umm_malloc itself.

Since then, I copied and modified the heap code from umm_malloc to set up a second heap in the 64k of previously-unused RAM (see this topic). Both heaps work correctly with a block size of 8 and with their respective allocation counter variables. This new heap's code is in modules/vgm/vgm_heap2.c . The new heap will seamlessly start allocating data in the normal heap when it runs out of space. I would suggest this code be usable for other projects, but there's a major drawback of this second heap: the RAM it's stored in is inaccessible to DMA operations. Which means that you have to know whether any memory you allocate at any given time will be filled via DMA (e.g. reads from SD card, etc.). So for now this second heap will only be accessible to MIDIbox Quad Genesis.

Hi TK., Since my project MIDIbox Quad Genesis uses dynamic memory allocation in C, I need to know at all times what percentage of the heap is in use. The umm_info function in umm_malloc.c gives this information, but it takes a couple hundred milliseconds to iterate through the heap (and send a ton of debug information to the terminal), and this causes the synth to freeze for this amount of time which is unacceptable. So instead I modified umm_malloc and added a variable, umm_numusedblocks, which gets updated every time umm_malloc, umm_free, and umm_realloc are called. It just keeps track of the changes in the number of blocks allocated, it doesn't traverse the whole heap. (The additional overhead is a couple of adds and 4 bytes of memory.) After some debugging, this system works in my synth. I created some test cases to make sure the block count increases and decreases correctly in the cases of malloc and free, and also in the cases of realloc to a larger and a smaller size. In the process of working on this, I discovered that heap blocks are actually 12 bytes, not 8 bytes like the comments say. This is because (the size of the block is defined in umm_malloc.c at line 564) the union 'header' is 4 bytes and the union 'body' ends up being 8 bytes because portBYTE_ALIGNMENT is 8 bytes. Now, obviously this system is working for all MIOS32 projects, but it doesn't look quite right to me. In fact it looks like someone changed the size of data from 4 (or maybe sizeof(umm_ptr) or whatever) to portBYTE_ALIGNMENT, thinking this would cause all addresses returned by malloc to be aligned to 8 bytes. There's three reasons this isn't true: Obviously, this makes the blocks 12 bytes, which means every other one starts on an 8-byte boundary; so on average half the returned addresses won't be 8-byte-aligned. Even if this wasn't the case, the block has 4 bytes of header data, so if the block was aligned to 8 bytes, the returned pointer would be aligned to 4 bytes. Besides all that, there's no guarantee the heap itself is aligned to 8 bytes. In fact when I first defined umm_numusedblocks, I made it unsigned short int (i.e. u16), because this is how all the blocks are addressed in this library. It turns out that this caused the heap itself to get aligned to a 2-byte boundary, which caused mallocs to fail and the synth to go up in flames. Now it's an unsigned int (i.e. u32), so the heap is aligned to 4 bytes, but I can't guarantee it'll be aligned to 8 bytes. Moreover, I don't know of anything in ARM32 which would need to be aligned to anything larger than 4 bytes. And in fact, we know there isn't, because the current system isn't aligning half of its contents! Therefore I tried changing data[portBYTE_ALIGNMENT] to data[4], and it seems to work fine. My synth runs fine, and in fact uses 0.1% less memory this way due to the smaller block size. :) And I also compiled MIDIbox SEQ V4 with this modified umm_malloc.c, and it starts up and appears to run just fine. So I would like to commit my modifications, so that I can continue using this code in my synth (and maybe other people can benefit from the RAM usage tracking too). But I wanted to get your approval before modifying a critical part of MIOS32 for everybody. Would you like me to commit it somewhere else and you can test it first? Or do you just want me to commit it normally, and you'll test it in place, and if there's an issue you can roll back the files with SVN? Thanks, Sauraen

I fixed up your section and added a little more, I will add even more later. :)

I would just say, the dream of having a complete library for all your needs and just popping in the parts you need is very difficult to realize. At work we use Altium ($8000 per seat, private server, etc.) and we only barely manage this (using Subversion for library management). For every new board I work on there's at least half the parts which have to be made from scratch. And we have a strict policy that all the metadata must be complete, all parts must have correct 3D models, etc. before being used; so it usually takes longer to make the parts for a board than to lay out the board. That said, maybe a guide to using KiCAD, with tips, would be helpful. There's nothing I haven't been able to do with the program, but there's a number of things for which I've needed extensive workarounds. I would be happy to contribute to this if someone else gets it going.

No idea what I just watched (was that just an elaborate music video? in that case the video was much better than the music...), but that was cool!

Okay, that answers that question, thanks! I don't know how you dug this up, but that's very helpful.

It all depends on if people start using MIDIbox Genesis Tracker with FMDrive--if they do, I'm sure we'll get bug reports that things don't sound the same, and then go from there. I do have that file, that's what I based the implementation on. It still has issues (CC 92+93 are assigned to two different things, not to mention the whole velocity problem), but there's also one major thing I couldn't figure out. For a parameter that's let's say 4 bits, does it use the lower 4 bits of the CC value or the upper 4 bits? If the lower ones, does the range clamp or repeat? I have it set up now with a macro that can be defined either way, but I will obviously need it to be the right way for a MIDI to work.

That would be great, if you can get me in contact with either of them privately also. I use REAPER, sorry no Ableton/Max. From looking at the docs it seems that GenMDM was buggy and never fully supported... ouch. My implementation may end up varying in small details in order to be more consistent/correct. Also, since I've been part of the effort to fully reverse-engineer the YM2612, I've become a bit of a snob about emulation accuracy, especially since some of the undocumented features are used in MIDIbox Quad Genesis. It looks like Aly has a handle on some of these features, e.g. CSM mode, but I'm not sure whether to trust his implementation of SSG illegal modes or the test bits.

Oh, I know all about JUCE, and I used to be an arranger for vgmusic. But all the MIDI files I've seen there are intended for playback on computer MIDI synths (MSGS Wavetable SW Synth), not on custom hardware. A GenMDM-compatible MIDI would completely not work or sound terrible on a normal MIDI synth, and vice versa. I knew about this VST, though I didn't know it was GenMDM compatible. I'm more looking for a set of MIDI files that use this standard, so I can play them into the synth and make sure all the commands are working. The GenMDM spec isn't exactly clear on a few points. Maybe I should contact Aly James or Littlescale directly?

Thanks everyone! Now I'm getting down to programming the actual functional modes. Regarding MIDIbox Genesis Tracker, the idea was that it would be a GENMDM-compatible backend that you could stream MIDI commands into to play the sound chips (like the GENMDM cartridge for the actual Genesis). The code for this is actually written and partially tested. However, I actually don't know anything about Genesis trackers and other PC software, so I don't know what software would be used to play into GENMDM or MIDIbox Genesis Tracker. I also don't know where to find GENMDM-compatible MIDI files that I could play into the synth. Anyone know about this?