robinfawell Posted March 29, 2004 Report Share Posted March 29, 2004 Although this query relates to my organ project, this is likely to be of general interest.QUERY I believe that I can write sysex files to a Bankstick using the MIOS Sysex implemetation commands, using command 03 and then Command 02.To test that the files are stored I would then, using Commands 03 and 01, read from the Bankstick.I would use MIDIOX to send and receive the data files.Is this procedure correct? Do I need to prepare the Core module other than giving it an ID and programming the bootstraps, MIOS and Midio128?I do realise that ultimately the send routines to transmit the sysex messages to my sound module will be controlled by a button handler.Regards Robin Fawell Quote Link to comment Share on other sites More sharing options...
TK. Posted March 30, 2004 Report Share Posted March 30, 2004 Yes, in principle thats correct.There is a tutorial application in the MIOS download section which explains BankStick transfers step-by-step, search for "bs_xfer_v1_4.zip"Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 5, 2004 Author Report Share Posted April 5, 2004 Dear ThorstenThank you for the last reply.You referred me to the BS Transfer Method. In Step 1 you state that the file Main.asm includes P=EEPROM16. It contains "LIST P=PIC18F452, R=DEC". There is no reference to EEPROM16.I am not ready yet to modify the main.asm file and it seems easier (for me) to use the MIOS Sysex implementation method.It might be better if I explained in detail what I am trying to do.I have recorded using Midiox several sysex messages. These are "setup" instructions for the Roland SC8850 Sound Module. I want to store these in my Bankstick module (0-7). This is built but not tested.ChecksumIn the sysex implementation text page 3, it refers to "checksum: inverted checksum from ah to the last byte of <dump>.In hex to Sysex converter you refer to "-(sum of bytes) & )x7f. Are we counting 8 bit or 16 bit bytes?If we count the bytes fom ah to the last dump byte in binary invert the result ie 0101 becomes 1010 and then "AND" the result with 7f (0111 1111) would this be correct?Do we always have to use perl to convert to syx from hex to send midi messages via Midi in?Thanks Robin Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 5, 2004 Author Report Share Posted April 5, 2004 I left this out of my preceding message.In checking through the Sysex Implementation text under the command 01 examples I think there may be an error in the sysex message for the Internal Flash.. Â I believe that the ch parameter should be 0x20.If Counter = 0x08000 When this is shifted 10 Â and "AND"ed with 7f the result is 0x20 Â (binary 0010 0000) not 0x10. Â Of course I may be wrong and misunderstood the basis of the calculation.Regards Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted April 5, 2004 Report Share Posted April 5, 2004 Hi Robin,you are right, I forgot to update the documentation about the EEPROM16 type, it must be PIC18F452 - EEPROM16 doesn't work due to an imperfection in the MPASM assembleryou are also right with the documentation error concerning the internal flash readout counter - ch must be 0x20But regarding the SysEx upload: it isn't possible to write any plain MIDI data (like SysEx strings) into the BankStick, they have to be encoded by the hex2syx script to another format. It's too complicated to convert an existing data dump into the MIOS upload format without this script. So, what you have to do: you have to write a script which converts your binary files to assembler text, so that the assembler can generate a .hex file, which can be thereafter converted to a .syx file by hex2syx.plThe assembler format can be found in bankstick_content.incBest Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
TK. Posted April 5, 2004 Report Share Posted April 5, 2004 Hi again,I've updated the bs_xfer package (MIOS Download -> bs_xfer_v1_6.zip)It now also contains a script which allows you to convert a plain binary file to assembler text, it can be found in the tools directory (tools\bin2asm.pl)So, here the flow to bring plain binaries into the BankStick:use bin2asm.pl to convert your binary file to assembleryou can copy&paste several assembler files to a single assembler file. Only requirement: the overall number of bytes should not exceed the 32768 boundary (0x407fff) if you are using a 32k BankStick, or 65536 (0x40ffff) if you are using a 64k BankStickuse MPASM to get a .hex fileuse hex2syx.pl to get a MIOS upload fileYou will notice that this conversion process simplifies the creation of the BankStick content a lot, since you are able to move and rearrange the converted binary files with a text editor to free BankStick addressesBest Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 5, 2004 Author Report Share Posted April 5, 2004 Thanks Thorsten for such a fast response.May I ask you to clarify the checksum queries.Regards Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted April 5, 2004 Report Share Posted April 5, 2004 Hi Robin,I thought that the checksum question became obsolete since you will use the hex2syx.pl script which calculates the checksum automatically.Maybe you've misinterpreted the term "sum of bytes". It should mean: the checksum is the addition result of all bytes within the address/counter field and all bytes of the dump field. Than it will be negated, so that the checksum within the SysEx file + the calculated checksum (which will be calculated by MIOS during the upload) results to zeroThe checksum is a 7-bit value, therefore only the first 7 bits are used (thats the "& 0x7f")Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 6, 2004 Author Report Share Posted April 6, 2004 Dear ThorstenThanks for straightening me out.I was in the middle of assembling a binary file complete with 01, 03 commands.I am going to practice with my largest file, "reset" which will go to BS 00.I will generate a reset.inc file using bin2asm.pl.I will use MPASM to generate a .hex file.I will convert this to a syx file using hex2syx.pl and then upload this using Midiox.MPASMDo I use the default settings?I assume the source file is Reset.incProcessor EEPROM or 18F452?Which generated file?Tab size 8 ?Thanks again Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted April 6, 2004 Report Share Posted April 6, 2004 Hi Robin,I would suggest to create a copy of the main.asm which which comes with the bs_xfer package, because it already contains all required settings (and therefore documentates the requirements and answers your questions)From main.asm you can include one or more .inc files which contain the converted binaries.Hint: after the assembler has been passed, you will find the address offsets to the converted binaries in main.lst - they are starting at 0x400000, the last 4 digits (16 bits) are equal to the BankStick addresses.Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 7, 2004 Author Report Share Posted April 7, 2004 Dear ThorstenI have received the following response from the core module via Midi out after sending a test  .syx file.F0 00 00 7E 40 00 0F 07 F7 F0 00 00 7E 40 00 0F 7D F7 F0 00 00 7E 40 00 0F 3A F7 F0 00 00 7E 40 00 0F 6E F7 F0 00 00 7E 40 00 01 F7 The file was sent to BS 7.  I have worked out that the messages are to do with WREG. The last is a request for an upload, I think.  Does this message seem OK?Regards RobinPS device ID is 0x00 Quote Link to comment Share on other sites More sharing options...
TK. Posted April 7, 2004 Report Share Posted April 7, 2004 Hi Robin,yes, the whole upload procedure was successfull:> F0 00 00 7E 40 00 0F 07 F7 is an acknowledge message (0F) which reports that the received checksum matched with the calculated checksum, that the data has been copied into the BankStick and that it has been verified (read-back and compare)The value behind 0F (here: 07) is the same checksum like reported by the hex2syx.pl script. You will receive an error message (0E) if any step fails (e.g. BankStick not connected, checksum error, verify error, etc...)> F0 00 00 7E 40 00 01 F7is the upload request of the first level bootstrap loader.A reset instruction will be issued after up- or downloads via MIOS, therefore this message appears like on a common startupSo - now you are sure that the binary has been copied into the BankStick.Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 8, 2004 Author Report Share Posted April 8, 2004 Dear Thorsten I am progressing slowly but have reached another difficulty.So far I have uploaded a large and medium converted binary file to BS 0 and 7.The large file "Reset" bin file was recorded as 16470 bytes by Midiox. After conversion to Reset.syx the file size measured by Midiox was 59025 bytes.During the upload Midiox received 51 Sysex acknowledgements. .For the medium file the respective bytes were 960 and 3560 with 5 ack.As you stated in your last message it seems as if data is being stored.I have tried many different ways to read the stored data using the sysex read procedures with Midiox with no success. Is it possible to use the sysex Read 01 Command for this task?I am sorry to trouble you again so soon.Regards Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted April 8, 2004 Report Share Posted April 8, 2004 Hi Robin,it's normal that the resulting file is larger than the original binary after the hex2syx conversion -- this is due to the required conversion algorithm which converts 8-bit values to 7-bit values. MIOS converts them back to the 8-bit values during the upload.So, normaly you should get about 1184 bytes per 1k block (original size: 1024 bytes)But it's very strange that the converted .syx files are much larger.How did you convert the binary exactly?What is the content of the resulting assembler file? You should see the address offsets within the files (they are inserted as comments at the end of each line), what is the last offset value?Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 8, 2004 Author Report Share Posted April 8, 2004 Thanks ThorstenI recorded the sysex files using MIDIOX. I saved the data from MIDIOX as a .txt file.I created an .asm file from the .txt file using bin2asm. Using MPLAB I made an .hex file using the .asm file as source file. Note this is the only file I used. I converted this file to a .syx file using hex2syx.I used MIDIOX to send the data to the Core via Midi in.The last offset on the "Reset" fle is 0x40C0A0. Â There is a change in the format up to 0x4080B0. Â The data up to that point was db followed by eight double bytes ;; and then the address eg 40000 (1st add).After this there are numerous db data sets per line.As if the line return had been lost.Regards Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted April 8, 2004 Report Share Posted April 8, 2004 Hi Robin,I saved the data from MIDIOX as a .txt file.This could be the reason why the converted .syx file is more than 3 times larger than the original file.You've to save the SysEx data in binary format - just like a common .syx file. Thereafter use bin2asm.pl to convert this binary to a assembler file. Check the content of the generated assembler code, the data behind the db statements must match with the data displayed in the MIDI-Ox window.Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 8, 2004 Author Report Share Posted April 8, 2004 Dear ThorstenFollowing your guidance I have produced a new reset.syx file. Originally as recorded with 16470 bytes after conversion to a syx file now 20137. (both measured by MIDIOX.By the way with MPLAB I did not use main.asm with a Reset.inc file. I made a reset.asm and used it (only) as a source file. Was this OK?Can I use Command 01 to read the BS? As you probably realise I keep thinking that the MIOS system will respond to Midi sysex messages on Midi In. Plese let me know how to proceed.Thanks Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted April 9, 2004 Report Share Posted April 9, 2004 Hi Robin,alright - the size of 20137 sounds correct, these are 17 converted 1k blocks, and the original file was a little bit larger than 16k - perfect!Yes, now you could read out the BankStick again and convert the resulting .syx to a .asm file like described in the bs_xfer package. However, it's more like a toy, but it helps you to 100% ensure that the file has been uploaded.For a first test it's ok when you assemble the reset.asm file directly, but later you propably want to have multiple files in a single BankStick, and for this it would be easier to include all data files into a single assembler file, so that the assembler can take care about the data locations.Ok, now you should proceed with the function which sends the SysExstring, see:http://www.midibox.org/cgi-bin/yabb/YaBB.cgi?board=mios;action=display;num=1078174868Later you should think about how to organize the data. I would suggestto create a table which contains:the bankstick numberthe offset addressthe size of the SysEx stringBut I fear that I don't have enough time to explain more details before my holidays... :-/Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted April 14, 2004 Author Report Share Posted April 14, 2004 Dear ThorstenI can see the point of having multiple sysex strings held in each bankstick and being uploaded in one operation.I have tried many times using MPlab to include the .inc files into the.asm file without any result.  The build is signalled as successful but I finish up with the original .asm file.I note that each file (.asm or .inc ) starts with Address 400000.  Should not the .inc files  start with  different address(es)?There are also comments in each .inc file re deleting the line where there is a LIST instruction.  Should these be followed?Please confirm that I should be using MPLAB to make the composite .asm file and advise  any other instruction needed.Regards RobinSince sending the above message I wondered whether I should be composing the composite "write to bankstick" by cutting and pasting within MIDIOX. Quote Link to comment Share on other sites More sharing options...
TK. Posted April 26, 2004 Report Share Posted April 26, 2004 Hi Robin,did you solve the problem in the meantime? Root cause: overlapping data areas.You've to remove the "org" statements and to add a single one (which points to 0x400000) to the main.asm, or you have to relocate the data blocks by hand to free addresses. Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted May 2, 2004 Author Report Share Posted May 2, 2004 Dear ThorstenI have removed the org statements from the .inc files and placed a .org statement in the main.asm file. I have removed the LIST statement re PIC 18F 452 and the END from all three .inc files. I showed 3 include statements (for reset, trems and tremf) in the main.asm and removed the bankstick content .inc line.I have produced a main.lst file which to me looks promising. The reset .inc was 16k+ bytes and the trems and tremf .inc file were both 2.5k+ bytes. The main.lst file lists the three files however the list still shows the three files all originating from address 400000 admittedly with ;; in front.The problem is that when I try to convert the main.hex fle to main.syx I get no result other than a line showing the directory. (There is a main.syx file produced but it is blank) I have used the same procedure with a much smaller .hex file and it seems to work.I should add that the main.hex file is about 65k bytes.Please help.Regards Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted May 2, 2004 Report Share Posted May 2, 2004 Hi,just send me your project directory so that I can search for the reason.Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
robinfawell Posted May 3, 2004 Author Report Share Posted May 3, 2004 Dear ThorstenI have sent you 2 emails. One has an attachment with the project diretory file.The 2nd contains an outline of the organ project.Regards Robin Quote Link to comment Share on other sites More sharing options...
robinfawell Posted May 18, 2004 Author Report Share Posted May 18, 2004 Thanks Thorsten,I have been on holiday for the last week or so. Â The latest main.lst looks correct.I am still unsure whether the MIOS system will send the individual files stored in the BS to the Sound Module correctly.If I use the procedure outlined in http://www.midibox.org/cgi-bin/yabb/YaBB.cgi?board=mios;action=display;num=1078174868Will I get the file transmitted as I originally recorded by Midiox ? I know that using the Sysex instructions via Midiox produces altered files and increases the file size.. Â In the tutorial you state that a 32k BS produces 39k file size.Please comment.Regards Robin Quote Link to comment Share on other sites More sharing options...
TK. Posted May 19, 2004 Report Share Posted May 19, 2004 Hi Robin,somewhere is a misunderstanding, but I don't know where exactly, so I just try to explain the transfer flow again in short words:you've a binary file, let's say abc.syxwith bin2asm.pl you convert it to assembler lines, several binaries can be combined to a single assembler program in this waywith MPASM the code will be assembled to a single data dump, the result is a .hex filewith hex2syx.pl you scramble the data dump into a MIOS specific and especially MIDI compatible formatMIOS descrambles the data stream back to the original binary format during the uploadAfter the transfer the transfered memory content is the same like specified in the .hex fileYou could read back the data and descramble it with syx2asm.pl to check the BankStick content. This procedure is explained in the bs_xfer example...Why it is required to scramble/descramble the data during the MIOS upload? Because the binary data contains 8-bit values, but the MIDI spec only allows to embedd 7-bit values into a SysEx stream. Therefore hex2syx.pl rearranges the data so that no bit will get lost (you will notice that all values within a SysEx stream are between 0x00 and 0x7f, and never >= 0x80) and MIOS rearranges the bits again to reconstruct the original 8-bit values.Without this (clever) mechanism it wouldn't be possible to upload any data, not only the BankStick content, but also the flash and EEPROM content. Therefore it's proven.Best Regards, Thorsten. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.