An answer to the extended databanks problem

[stryd_one]

Hey crew. I nutted something out today which I hope will be helpful. It's been a M****F******** thorn in my side for the entire coding of the vX so far, so I'm stoked that I've nutted it and I hope this long post will help you avoid the same pain. I've thrown in a couple of other memory handling and performance specifics that came up along the way too.

<Ramblings of a tormented developer>

Most of you would have come across the "maybe it works, maybe not, nobody is too sure WTF it does" method of storing large sets of data (>256), by combining databank entries in the linker script. Those who have tried it, will find that it sometimes works like a charm, and sometimes not. Here's why, with code snippets from the vX as examples of what not to do hehehe

OK the example will be the track struct. I won't post the code for it here just yet, but suffice to say, it's big and complex with different types all through it, 100% yuck.

Now with no pragma marks set, the compiler will name the variable "udata_<filename of declaration>_n". You can see it in output\main.asm like so:

udata_main_1	udata
_track	res	314

//////////// See Note Below that 314 is the number of bytes used in decimal. It's two tracks, each 157B. Obviously they can't fit in a standard bank, so I extend the first bank in the linker:

// DATABANK   NAME=gpr0       START=0x080          END=0x0FF
// DATABANK   NAME=gpr1       START=0x100          END=0x1FF

DATABANK   NAME=gpr01       START=0x080          END=0x1FF

The result is that the struct is stored in that space, which can be seen in the project.map file:

             udata_main_1      udata   0x000080       data   0x00013a

All good right? Well... That depends. Let's say we have a for loop which goes through all the tracks playing step 0 (of course you wouldn't do that but just for example)

for (i = 0;i < 16;i++)  {
MIOS_MIDI_TxBufferPut(track[(i)].step[0].param1)
}

If you do that, then the variable could point to different addresses in ram, depending on the value of 'i'. Microchip direct that "Indirect addressing is a mode of addressing data memory, where the data memory address in the instruction is not fixed."  - and as this is the case here, the compiler will use indirect addressing. This works by calculating the memory location (aka pointer math) filling two registers with the memory address which was calculated and then finally reading from that address. This method of memory access does not require bank selection, but setting the address can be a costly process. Here's an example where the pointer math results in an address stored in r0x00 and rx02:

	MOVFF	r0x00, FSR0L    ; Low byte of the address
	MOVFF	r0x02, FSR0H    ; High byte
	MOVFF	INDF0, r0x00    ; Move byte from indirect file access register to temporary register
	MOVF	r0x00, W         ; Move to W and send the byte via midi.
	CALL	_MIOS_MIDI_TxBufferPut

Check out section 4.12 in the 452 datasheet for more, but the news here is all good. Because the memory of the PIC is seen by the indirect addressing scheme as one great big pool, it does not have any problem with crossing over the 256B banks, and if multiple bytes in successive addresses need to be read, the pointer math only gets done for the first address, and then the pointer can be incremented or decremented automatically. See 4.12.1. Let's say I have these two lines in the code to initialise the note number that the first step in track 0 and 1 should play.

        track[0].step[0].param1 = 0x50;
        track[1].step[0].param1 = 0x40;

In this case, the memory location does not change for these functions, and so banked memory access will be used. The ASM output looks like this:

;	.line	100; main.c	track[0].step[0].param1 = 0x50;
	MOVLW	0x50
	BANKSEL	(_track + 20)
	MOVWF	(_track + 20), B

;	.line	101; main.c	track[1].step[0].param1 = 0x40;
	MOVLW	0x40
	BANKSEL	(_track + 178)
	MOVWF	(_track + 178), B

Now that is all good... (_track + 20) is within the bank (it starts at 0x080, plus 20 decimal = 0x094, and the bank ends at 0x0FF. The variable in track 1 however is at 0x132, which is in this next bank. This is not a problem because of the bank select instruction before each read. And here's the twist - the ASM doesn't really look like that at all. What it looks like is this:

;	.line	100; main.c	track[0].step[0].param1 = 0x50;
	MOVLW	0x50
	BANKSEL	(_track + 20)
	MOVWF	(_track + 20), B

;	.line	101; main.c	track[1].step[0].param1 = 0x40;
	MOVLW	0x40
; removed redundant BANKSEL
	MOVWF	(_track + 178), B

Notice that the second bank select is gone. This is due to the compiler optimising switch

 --obanksel=2

If you change it to =1, the banksel's will be left in place, and the code will work. If you don't, then all the memory reads will return 0 regardless of their value, and writes will not work at all. Now don't get me wrong, that switch is a good thing. The banksel instructions waste MCU cycles, and we want them gone... Unless we are addressing a variable which crosses the border of a memory bank. Thanks to all of you who gave me the hints I needed to spot this, and for everyone involved in MIOS studio and it's SRAM read function without which solving this would have really sucked. //////////// Note: While you're looking at that register, you may notice something like this:

 ; Internal registers
.registers	udata_ovr	0x010
r0x00	res	1
r0x01	res	1

these r0xnn variables are registers used as temporary quick ram by the compiler. There is more info in section 4.10 of the 18F452 datasheet. This coincides with TK's advice to put frequently accessed variables in the access RAM, which does not require bank selection and is therefore faster to read and write. The access RAM has memory locations 0x000 to 0x07F available. (There is another access bank which is not used in the same way, it holds the SFR's which are used to control the PIC.) The MIOS Wrapper ensures that the first 16 bytes (0x000 to 0x00F) are reserved for MIOS Functions:

MIOS_VARIABLES		udata	0x0000
_MIOS_BOX_CFG0		res	1
_MIOS_BOX_CFG1		res	1
_MIOS_BOX_STAT		res	1
_MIOS_PARAMETER1	res	1
_MIOS_PARAMETER2	res	1
_MIOS_PARAMETER3	res	1
_TMP1			res	1
_TMP2			res	1
_TMP3			res	1
_TMP4			res	1
_TMP5			res	1
_IRQ_TMP1		res	1
_IRQ_TMP2		res	1
_IRQ_TMP3		res	1
_IRQ_TMP4		res	1
_IRQ_TMP5		res	1

The compiler will create the r0xnnn registers for use in your functions, and they are in the access ram also, starting from 0x010 as per the code at the start of this note. Hope this sheds some light on what's going on with those r0x000 things that keep showing up ;)

FWIW, you can use the available access ram with the usual #pragma udata and SECTION trick, but the compiler does not seem to be aware of it's nature when using that method and still bank selects before using it. Does anyone know a way around this?

Knowing all this is also very helpful in optimising our applications, because by using certain programming techniques we can optimise the ASM output by the compiler... Like, we can create structs and arrays in ways that will ensure linear access, and always use a variable when accessing them...

</ramble>

So, in summary:

Access: Fastest, very limited, for special purposes. Is there a way to get SDCC to use it properly aside from it's own common variables? (r0x00...)

Banked: Faster than indirect, only used for fixed addresses. Not suitable for extended banks - unless someone knows a way to turn off the optimizer for certain areas of code?

Indirect: Slowest because of the overhead required to load the memory location, but aside from those two instructions as fast as banked access when accessing multiple sequential bytes.

[audiocommander]

woooooooooosh!

(that was my head exploding).

But besides I have not really understood what's going on:

unless someone knows a way to turn off the optimizer for certain areas of code?

The SDCC manual just refers to the mentioned --banksel=nn option. Some options can be overriden by certain #define options in code. I don't think the banksel option is available (at least it's not mentioned in the SDCC manual).

Or you request a feature for implementation of a #define option.

If it's fixed so fast like last time, you might be a lucky one :)

cheers,

Michael

(are you going to revolutionise SDCC now ;D )

[stryd_one]

Well a #define would probably effect the whole file because it's pre-processor, so you could use --obanksel=1 to avoid it for the same effect -  in fact this is how I got the example code with both banksel's in it :)

BTW, if you do need to do that, don't change it in the makefile.spec: that will do it for all files and make big slow code; instead, run make, let it create the makefile and compile, then edit makefile, and change the switch only for the required files.

I was thinking of something like the __ASM flags, like __obankseloff and __obankselon... Maybe I should request it :)

[stryd_one]

Access: Fastest, very limited, for special purposes. Is there a way to get SDCC to use it properly aside from it's own common variables? (r0x00...)

Yes there is :)

If you declare with the __at keyword and specify a location within the access bank, sdcc takes care of the rest:

main.c:

extern unsigned char __at 0x030 myvar;

Produces project.map:

ustat_main_00      udata   0x000030       data   0x000008

....And code which accesses the ram without bank selection. Yay.

Developer General's warning - if you do this, choose your memory location wisely. :)

[stryd_one]

Warning: this topic has not been posted in for at least 120 days.

Unless you're sure you are a tormented developer, please consider starting a new topic.

;D

I'm still learning C... What can I say:

There is an easier and much safer way. Declare your variable using the 'register' keyword:

register unsigned char Foo;

This uses the Access RAM as intended, and allocates the address as required, allowing for the MIOS parameters. I guess you could say that this is the right way to do it.

A reminder that the Access Bank is small and for special variables, it's not a free pass to faster code ;)

[entity]

;D

I'm still learning C... What can I say:

There is an easier and much safer way. Declare your variable using the 'register' keyword:

register unsigned char Foo;

This uses the Access RAM as intended, and allocates the address as required, allowing for the MIOS parameters. I guess you could say that this is the right way to do it.

A reminder that the Access Bank is small and for special variables, it's not a free pass to faster code ;)

This is an interesting thread since I come from the "high-level language on x86 or newer processor" world... AFAIK, the register keyword "suggests" that the compiler stores the variable in a register when possible. If this variable is a pointer, given a pointer in a register, the compiler will probably choose to optimize using indirect addressing over pointer math (that's what your findings suggest). I'm curious what happens if the "suggestion" is not always consistent in SDCC (ie... what if you use more register pointers across multiple banks then the number of available registers on the PIC? Ya it is knitpicking....) I'm a noob with PICs and such, so sorry if this sounds like a silly concern.

Either way, I was running into this problem early and I think this is a clean solution in C. Do you still have to modify the linker script to combine the memory chunks?

[stryd_one]

AFAIK, the register keyword "suggests" that the compiler stores the variable in a register when possible.

....

I'm curious what happens if the "suggestion" is not always consistent in SDCC (ie... what if you use more register pointers across multiple banks then the number of available registers on the PIC? Ya it is knitpicking....)

In SDCC at least, the register keyword causes the variable to be stored in Access ram, which means you don't have to perform bankswitching to access the variable. At compile time, it will generate a warning to tell you that it has done this, and if you allocate more registers than there is access ram, it won't link after that.

given a pointer in a register, the compiler will probably choose to optimize using indirect addressing over pointer math (that's what your findings suggest).

They do? ;) Nah... That's about how the access ram works.

If you are using the data ram banks, it will have to use either indirect addressing (takes the location of the variable and loads it into FSRxL and FSRxH, then reads it using INDFx/POSTINCx/POSTDECx/etc) or by switching to the correct bank and reading it by name (banksel blah, Movlw Foo, etc).

Access ram allows you to just specify the variable's name, without having to load the pointer to the variable into FSRs, or switch banks ('movlw foo' will always work no matter what bank you are in or what values are loaded into the FSRs) MIOS_PARAMETERx is a good example.

SDCC always uses indirect addressing when pointers are implemented (places the pointer in FSRxL/H). And it recalculates the address every time you use the pointer :(  This is by design, so if you want to use pointers you should be careful about how you do it if code size and speed are a concern.

There's no need for a pointer to a register (access ram) obviously, and a pointer to another address which is stored in access ram will be slightly faster to calculate because of the lack of bankswitching/FSR loading, but given the overheads involved, it's not really worth your trouble.

Registers/Access RAM are really best used as a temporary variable which is fast to access, or perhaps to pass variables to a function without specifying them as an argument which would use the stack. (As Wilba called it, writing ASM in C ;) It's probably bad practice in C but it can make for faster code in the end).

I'm a noob with PICs and such, so sorry if this sounds like a silly concern.

Not at all! I took a while to respond because it's a fairly indepth topic so I wanted to take my time on it :) Hope this helps!

Do you still have to modify the linker script to combine the memory chunks?

That's a different issue... You only need to do that if you are trying to store a single variable which is larger than a whole bank - which is generally something that should be avoided.

[entity]

Thanks for the info stryd_one. You are a wealth of PIC info! Some of this is taking some thinking on my end... I thought I'd never have to learn ASM or worry about hardware. That's biting my butt now. haha When I get my PIC back I'll probably revisit this post a bunch.

[stryd_one]

... I thought I'd never have to learn ASM or worry about hardware. That's biting my butt now. haha

Hahahah

Well, I think that if there's a moral to this thread, it would be:

"This is how you can do it, if you really need to.... which you almost never should!"

If you find yourself needing to refer to this thread, chances are, you're doing it wrong ;D