Jump to content

vX


stryd_one
 Share

Recommended Posts

Oooh a new section in the forum!

I thought I should honour the occasion in some way so I thought I might share some of the research that I did for the subclocks in the vX. I'll try and make this make as much sense as possible and include lots of pretty pictures ;)

So here's the idea. I want to be able to run a group of clocks, which will be synchronised with each other in some way, but will trigger different numbers of steps per bar, like this:

:|x  x  x  x  |x  x  x  x  |:  8 steps/2 bars

:|x  x  x  |x  x  x  |:    6 steps/2 bars

The reason behind this is to have a sequencer which will allow the use of polyrhythms, which are widely used in ethnic music and the only way to play breakbeats in their original form (before western notation got involved and the boffins decided to make it fit into their existing timing scheme). If you're not sure what I mean, find some old "taiko" (sometimes called "daiko") drums and be introduced to my favourite form of music :) Polyrhythms are enjoying something of a resurgence in some forms of music in the last few years too (especially in industrial electronica and rock), and I suspect that we'll hear them on the charts before too long  ::)

So, how to go about this? .... Most of the time, a sequencer will count down a certain number of MIDI clock ticks before triggering the next step. Because MIDI clock ticks at a rate of 24 ticks per quarter note, for example, If you want 1/4 notes, you wait for 24 ticks between each step. For 1/16th notes, you want 1/4 of that time, so you wait for 6 ticks to pass before triggering the step. For a 1/4 note triplet, you would want to trigger 3 times every 1/4 note, so that's 3 times every 24 ticks, so 8 ticks. If you want a whole note (one bar long) then you would wait for four 1/4 notes, 96 ticks, to pass between each step.

It may be becoming clear to you now why the number 24ppqn was selected - at 96 ticks per whole note, it makes it simple to find commonly used measures of time in modern music, such as 1/2, 1/3, 1/4, 1/6, 1/8, 1/12, 1/16, 1/32 etc etc etc. If you take 96 (ticks) and divide it by any of those numbers, you will find that you get an integer (no decimal point). Awesome.

So what if I want to hear 5 steps per bar? This is not the same as 5/4 time, which would be 5 of the 24ppqn ticks - I'm talking about 5 steps in a normal 96 tick bar. If we divide 96 by 5, we get 19.2. Houston, we have a problem.

There's no such thing as 0.2 of a tick, and the countdown to the next step is measured in ticks. 0.2 of a tick is empty space between the ticks themselves. How would we know when to trigger the step?

Now, if we want to hear 5 steps per bar, we could wait for 19 MIDI clocks to tick between each step. But of course this means that after 5 steps, only 95 ticks have passed. This means that the next bar would start one tick before a normal length bar of 96 ticks would. And the next time it would start two ticks too soon, then 3, and 4, and before you know it, the bar is a 1/16th note out of sync with the rest of the song, and getting worse by the moment. No good.

Alternately, we could wait for 20 ticks each step, but then we get the opposite problem, and the same result - it's out of sync.

The trick here is to remember that music is ALL about fractions (ratios, if you like) and fractions don't use decimal points, they use a "remainder" or "modulus". So 1/5 of 96 is 19 with a remainder of 1. The remainder of any combination will be the amount of 'drift' in synchronisation explained just above.

So, no problem. All we need to do is tack the remainder onto the end of the bar. So 5 steps will be triggered, each with 19 ticks between them, and when that is over(95 yicks have passed), we wait 1 more tick (the remainder), before starting over. Obviously, it is not perfect, but it is as close as possible with the master clock (24ppqn) that we have. The first note will begin at the correct time, and the notes will be 0.2 of a tick too soon each time, until the last tick, where the remainder is added on, and that note is a little longer (19(normal length)+1(remainder)), to compensate for the others being a little too short. This error of 1 clock tick is not really much, and is pretty much acceptable. Humans have drifted more than this for centuries and nobody complained ;) Yay a solution!

Bzzzt wrong. There are two problems with this idea.

If this 1/5 note clock is started at the same time as a normal 1/4 note clock, then they will both start together and end together. Fine. But if they were started at different times, the remainder may get tacked onto the end of the 5 notes, but that might be played in the middle of a synchronised 4 note bar. In this case, the error is not really increased beyond that described above, so it is still a deviation of 1 tick, as good as we can get....but we should consider...

What if our remainder is bigger? For eg, What if we are talking about 9 steps per bar? (10.6 recurring ticks per step) Now we have 10 ticks per step, for 9 steps, which leaves us with a remainder of 6 ticks.  it would look like this:

Step  ticks

1        10

2        10

3        10

4        10

6        10

7        10

8        10

9        16! (that's 10, plus the remainder of 6)

Yuck! Imagine that big long step sitting in the middle of a nice neat 4/4 bar.... and the maximum error is now quite large too, because even after two steps there should be 21.333 ticks, we're already over one whole tick out of sync. after 3 steps, there should be 32 steps but there have only been 30, now we're two ticks out of sync. And it just get's worse and worse until we are finally 6 ticks out of sync. Bad.

What we want, is to slowly smooth this remainder out over time. In the chart below, we can see a way of doing this with floating point calculations, just for the purpose of illustration, again using 5 steps per bar. The floating point calculation is on the right hand side. It shows how many ticks would pass to divide the bar up evenly. The next column from the right shows the floating point number rounded to the nearest integer. the 'ticks' column shows the number of ticks inbetween those integer figures.

Step  ticks  FP, rounded  ticks (floating point - the ultimate and impossible goal)

1        11*    11              10.6`

2        10      21              21.3`

3        11*    32              32

4        11*    43              42.6`

5        10      53              53.3`

6        11*    64              64

7        11*    75              74.6`

8        10      85              85.3`

9        11*    96              96

Remember that our gap between steps was 10, with a remainder of 6? Notice that sometimes, if we round to the nearest whole number of ticks, we need to add 1 tick to the normal gap of 10 ticks. Notice that in one whole loop, that happens 6 times? That's our remainder, and if you look at the chart above, you can see that it is not just 'glued' onto the end of the bar, but nice and neatly distributed throughout the loop. MMM, pretty.

Now, that is all well and good, but as you may be aware, floating point divisions are very processor intensive. If we calculated the countdown between steps like that, well, it just couldn't be done .

So my theory was, that the distribution of the remainder, was directly related to the value of the remainder and quotient. And sure enough, it is. I just needed to find a computationally inexpensive method of calculating when to add a tick from the remainder, and when to not - Or looking at it from a floating point perspective, when to round up, and when to round down. And I need to do it by addition and subtraction and maybe multiplication, but definitely not dividing.

So I came up with this method of modulus distribution. I'll continue with the example of 9 steps per 96 tick bar.

When the clock is first initialised, there is a divide operation. This is the only occasion where a divide is required. This returns the quotient and modulus of the equation. (eg for 9 steps, the quotient is 10 ticks, and the modulus (remainder) is 6 ticks). There is a variable used to count down to the next step, and the trick is, a 'modulus counter' variable. Here's how it is used.

  • When the clock is initialised, modulus counter is set to the modulus (6).

    ...
    • the countdown until the next step is set to the quotient (10)
    • To calculate the length of the next countdown, the modulus (6) is added to the modulus counter (now=12).
    • If the modulus counter (12) is greater than the modulus (6)(it is), then 1 tick is added to the countdown (now=11), and then the quotient (10) is subtracted from the modulus counter (now=2)
    • The countdown is decremented each time a midi clock event is received
    • When the countdown reaches 0, the next step is triggered, and this process is repeated....



      • the countdown until the next step is set to the quotient (10)
      • To calculate the length of the next countdown, the modulus (6) is added to the modulus counter (now=8).
      • If the modulus counter ( 8 ) is greater than the modulus (6)(it is), then 1 tick is added to the countdown (now=11), and then the quotient (10) is subtracted from the modulus counter (now= -2)
      • The countdown is decremented each time a midi clock event is received
      • When the countdown reaches 0, the next step is triggered, and this process is repeated....



    • the countdown until the next step is set to the quotient (10)
    • To calculate the length of the next countdown, the modulus (6) is added to the modulus counter (now=4).
    • the modulus counter (4) is not greater than the modulus (6), so the countdown is left alone (still = 10), and the modulus counter is left alone (still=4)
    • The countdown is decremented each time a midi clock event is received
    • When the countdown reaches 0, the next step is triggered, and this process is repeated....




      And if you can actually make sense of that mess you may have noticed that there is a pattern formed - it goes 11, 11, 10, 11, 11, 10, ..... Just like the rounded floating point calculations! Yeehah!

      This all sounds more complex than it is.... it comes down to a matter of a few instructions each time a new step is triggered.

      Anyway I promised pretty pictures, and of course I had to test the algorithm out to see how it would work and how accurate it would be, and it's resulted in a big excel document with a couple of graphs. The first one is a graph of a chart like the one above, with the floating point calculations compared to the vX algorithm. It's extremely boring, just two straight lines. That's actually a very good thing, because I don't think you'd be able to tell them apart without the index:


      Boring. But nice to know.
      I've also graphed the deviation from the perfect result. To make sense of this, if the points ever reached the edge, they would be going beyond a whole tick out of sync, but so long as they are within the circle, we are as accurate as it is possible to be. Points at the center are perfect. But the reason for this type of graph is to demonstrate the rhythmic nature and even spread of the modulus. The end result is a flower. Oh how nice. I have attached the graphs for a 9 step bar over 1 and 4 bars.

      The more complex the subclock timing, the more interesting patterns come out of this as the algorithm struggles for perfection. Prime division makes for really interesting neverending flowers, and I included a 63 steps per 15 bar pattern graph just for kicks.

      Hope you like the happysnaps and the gory details  :D
      I'll follow up with more detail (this was a simplified explanation) and some posts on the rest of the sequencer as time goes on...
      accuracy.GIF
      dist9step.GIF
      dist36step.GIF
      dist6315.GIF
Link to comment
Share on other sites

Beautiful peace of work stryd_one!

I dimly remember a similar problem coming up in the design of some kind of digital oscillator, to do with the output bit rate and the number of instruction cycles available for each sub process. It was a long time ago, and I swore I'd never look at another one of those old TI DSP chips again. The notes are quietly mouldering away on a backup somewhere, but I can remember that the solution involved factoring for the mutually prime factors, and then a lookup table. The problem was in a similar domain to yours, no floating point allowed, and speed being very much of ther essence, both processor and project!

With the processors we've got here, there might be enough space for precomputed FLT's for all possible instances. How far up the count scales do you need to go? you'd be able to pull out the regular ones and then just FLT the mutual primes, I think.

I could just be barking madly up ther wrong tree here though.

Best wishes

Mike

Link to comment
Share on other sites

Actually the prototype is a bit different now, I have to convert the new documentation to wiki format so maybe check the link in the next few days :) Sorry for the poor timing! I'll post here when I update the wiki.

I should mention something I forgot in my FBAP, the vX does it's calculations in 96ppqn ticks for better accuracy...  I just did the examples in 24ppqn because it's nicer to read and the numbers are easy to understand :)

Mike thanks for your kind words and especially your analysis ; I'm just a hobbyist at these things and I've been secretly lacking confidence that my solution was optimal, so it's nice to have that double-check, and it sounds as though it's the voice of experience :)

I thought the same thing about the lookup tables when I saw the patterns, but there's a couple of catches... Because of my desire to dispose of linearity, there is no song counter which can be used as a reference. That's one of the bonuses of the algorithm, it operates in a way that allows it to stay in sync with other clocks without any awareness of it's relation to them or the 96ppqn master clock. There is the option of storing the rounding patterns (like 'up up down repeat', for 9 steps/bar) but the table ends up being very large to support all the different possibilities, and it also means that the time taken to calculate the next step will vary, meaning the timing could be unpredictable...

On a happy note, the scale engine from seqv3 is sure to be "borrowed" for the vX (which reminds me I need to finish off the tables this weekend!) so gamelan dreams await :) With full credit to TK, of course!

Stuffed if I can find that Ardley album though :(

Link to comment
Share on other sites

Stuffed if I can find that Ardley album though :(

http://www.musicsogood.com/duskfire/info1.htm

Which is good for me too - I've found a remastered copy, because all I have is the Vinyl. Then I listened to some of the samples, and my screen went all blurry, as I was mentally dragged back to first hearing an album I've not played since I moved, which is means I've not heard it for over 4 years. Ardley was a genius, it still sounds fresh now. As soon as I saw the re-master I bought it! Music has an emotional hit that is hard to explain sometimes.

It was a piece that got me into pattern loop work long ago, (well that and Terry Riley's stuff). I've not done any for over 20 years, and now I might know enough to make something worth keeping the recordings of! I do remember my earliest attempts at rhythm tracks as a kid in the 70's using spliced up tape loops. I'd get a sort of polyrhythm then usually submerged under a lot of tape noise!

It was your idea started me thinking. Thank you very much for that.

Algorithmic is better than FLT, I agree. I did have a thought along the lines of storing a precomputed seed in a table, and using that to generate the actual sequence, perhaps by PRBS. I need to get my PICing up to speed then I can make a more sensible contribution, rather than cheering from the sidelines. Presently I can usually follow code, but I think my writing is rather lame. There will be some kind of cyclic pattern eventually, though some could be a bit long - sudden horrible thought of someone trying to go all 'harmony of the spheres', (in the pythagorean sense, not another Ardley reference), and do 22/7! But I must take a look and see if there is any commonality with the 'comma of pythagoras' problem in tuning.

Thanks again

Mike

Link to comment
Share on other sites

I'm with smash on that one hehehehe

I'll do that post justice and read it after some work I gotta do, when I can pay full attention..

In the meantime, more pretty pictures. Accuracy of 22/7 over 10000 steps maxes out at 0.987, still within 1 master tick but pushing it's luck as the primes tend to do when divided ;) Naturally the error's growth is inverse exponential so it is still .987 at a couple hundred cycles. The second one is two cycles, to illustrate it's tendency to restrict itself from spiralling out forever, which it looks like it'll do in that first pic. Ain't it great to see the way each spiral gives birth to a new one when it is killed. I love mathematics, truly the stuff of life. :)

dist227.GIF

dist2272.GIF

Link to comment
Share on other sites

  • 1 month later...

some sound examples would also be very nice.

There'll be public alphas as soon as it can do that :) The vX is waiting for me t do some other things (seqv3 scales!) but I could hardcode a sequence into it to demonstrate polyrhythms if you like?

but looks like a very strange tool.

Thanks! I'm tired of the same old music :)

Link to comment
Share on other sites

  • 2 weeks later...

I'm very happy today, not just because MBSEQv3 is out, but because I've finished testing the subclocking engine.

The clocks work a treat, I've thrown everything at them and they always come out bang on time.

It's only taken me a couple of weeks (of a few hours on most nights) to go from the pseudocode to the working software, and I'd say at least 50% of that is attributable to my running into a bug in SDCC (allocating memory for arrays. 2.6.0 is fine, nightlies are currently broken) and to being a total n00b and doing some dumb sh!t because I'm new to C, which I've learned from and won't make again..... So obviously my preparation helped a lot, in fact most of the calculations were a copy&paste from the pseudocode hehehehe

Ahhh polyrhythms are my bitch now ;D ...and polymeters are my gimp. There's still a long way to go, but this was a milestone and I had to share my excitement :)

Link to comment
Share on other sites

Well funny you should mention it, cause I went to make a demo and realized that a recent addition to the design introduced a problem that could be tricky... Handling Note Off events... :( D'oh

<Ramblings of a tormented developer>

Here's the thing... Master clock ticks (IE MIDI F8's or a timer) increment counters until they hit their limit, and send a tick to the subclocks. The subclocks (now) have a patchbay, for routing the ticks to any number of tracks... So when a step is triggered, how do we reference the length of the note?

I've got lots of ideas but at the moment, it's undecided... If you're feeling imaginative... heheheh. Keep in mind that these might not be a Note Off event, but might be something like a command to mute a track (So you can use a step in one track to play another sequence of notes)

A happy answer to this post would be good, then I can reverse-lookup the patchbay routings... of course then I have to figure out which subclock to use as a reference if there are more than one routed to the track, or finding a quick way to calculate a useful reference from a number of different step lengths...(Edit: that's not gonna happen. Thx AC :) )

Because of that, I considered just allowing to assign a subclock for each track to use as a reference, however that could mean dropped notes if that subclock is ticking slower than the clock which triggers the note-ons... And then what should the default be? I don't want to interrupt the flow of using the seq to question the operator if they want to set this subclock to be the reference for lengths, every time they change the subclock routings - and can't, when the patchbay is altered by an internal function.

I think what I'll do is refer to the master clock by default, and allow the setting to be changed to reference any of the subclocks

Argh! This whole modular design has turned out to introduce exponential levels of difficulty with every feature ::) This post is just a glimpse of it...

</selfinflicted>

While I'm asking for peeps to make a request on the implementation of note off (etc) events, I should also ask...

What kind of control functions would you like to have? There are obvious things like sending notes and CC's to the outputs, or wierd sh!t like the abovementioned muting/unmuting tracks, changing notes on other tracks, changing track dividers, changing subclock routing to tracks, changing the current step on a track, etc etc etc... If you think of anything handy shout out :)

Link to comment
Share on other sites

Derr!

<Ramblings of a tormented developer>

Why not just pick a reference clock for each step rather than the whole track?

More info: the idea here is that each step will have a multiplier/divider of it's reference clock. So let's say the clock used as a reference is the master clock, there will be 96 master clock ticks as the base reference, and that can be bitshifted four bits in either direction to give 1/16, 1/8, 1/4, 1/2, or 2*, 4*, 8* 16*.

This number is then used as a counter, and when the counter overflows it empties a buffer with the note off (or whatever) data. It's just like the MBSeq.

SO the point of this, is that it has to use a reference clock, but it won't make any difference in processing to have the reference set per step rather than per track, because it will always have to fetch it. :)

</sleepisgood>

Man that was daft. This is why you shouldn't make important decisions when you have a fever hehehe

Link to comment
Share on other sites

Just wrapping up the note off handling now :)

<Ramblings of a tormented developer>

Wasted a day discovering an interesting behaviour, maybe someone could shed some light on this for me... (Developers take note: this could help you one day)

I've got this really big struct, for the clocking system, so I have to merge two banks in the linker.

I end up merging them all into one. 1

// DATABANK   NAME=gpr0       START=0x080          END=0x0FF
// DATABANK   NAME=gpr1       START=0x100          END=0x1FF
// DATABANK   NAME=gpr2       START=0x200          END=0x2FF
DATABANK   NAME=gpr       START=0x080          END=0x2FF
I use the udata pragma mark to force the struct to be stored in that bank. 2
#pragma udata gpr mclk
extern volatile mclk_t mclk;
I then make another big struct, for the tracks and steps. I write a function that uses that struct. My function is not working. Grrrr. I bugshoot for ages (like 6 hours) until I discover that the first step of the first track is ALWAYS 0! I try a few things, and by accident find that the declaration of the struct seems to have an effect on the data stored in it. namely I changed from this:
typedef struct _track {
	unsigned char frststep;  //First Step 5bit (Number of the step to play first)
	unsigned char laststep;  //Last Step 5bit (Number of the step to play last)
To this: (With the two top lines swapped)
typedef struct _track {
	unsigned char laststep;  //Last Step 5bit (Number of the step to play last)
	unsigned char frststep;  //First Step 5bit (Number of the step to play first)
And now the frststep value is correct but the laststep value is always 0. WTF, right? So I start thinking maybe something has gone awry in the allocation of the RAM, so I check the map file and....
                      gpr      udata   0x000080       data   0x00007f
             udata_main_0      udata   0x0000ff       data   0x000102
The gpr is the clock struct I forced in there. The udata_main_0 is the track struct, which has been allocated it's ram by the compiler. I then noticed... The compiler had simply gone for the next available byte in the one and only bank availabe -my track struct was beginning at 0x0FF ....right at the END of one of the 256b blocks. No matter what variable I put in that first byte, it would always be zero. I currently have changed it to this:
DATABANK   NAME=gpr0       START=0x080          END=0x1FF
DATABANK   NAME=gpr12        START=0x200          END=0x2FF
With no sections, and no pragma markers in the code. It maps out like this:
             udata_main_1      udata   0x000080       data   0x000102
           udata_mulint_0      udata   0x000182       data   0x000002
           udata_mulint_1      udata   0x000184       data   0x000002
           udata_mulint_2      udata   0x000186       data   0x000002
         udata_vxclocks_0      udata   0x000188       data   0x000001
         udata_vxclocks_1      udata   0x000189       data   0x000001
         udata_vxclocks_2      udata   0x00018a       data   0x000001
         udata_vxclocks_3      udata   0x00018b       data   0x000001
         udata_vxclocks_4      udata   0x00018c       data   0x000001
         udata_vxclocks_5      udata   0x00018d       data   0x000001
         udata_vxclocks_6      udata   0x00018e       data   0x000001
         udata_vxclocks_7      udata   0x00018f       data   0x000001
         udata_vxclocks_8      udata   0x000190       data   0x000001
             udata_main_0      udata   0x000200       data   0x00007f

udata_main_0 is the clock struct, and udata_main_1 is the tracks. notice that small variables used throughout the app now fill the spaces between the banks.

This got rid of my 'always zero' variable right at the top of the first track, so that I could continue coding, but I do fear that the 256b boundares between banks can not be crossed successfully, and that one I enable all the tracks and clocks, the annoying feature might rear it's head again...

I hope someone can make sense of all that for me!! You can imagine that it was not an easy one to track down, especially when you're like me and more inclined to blame your newbie-ass C code than linker issues. If I could know enough to avoid it in the future that would be really reassuring!

Notes:

1) Mistake #1

2) Mistake #2

</justshootmenowmun,justeasy>

Link to comment
Share on other sites

Stryd_One,

Looks like you had a long night. :o

Bank selection is troublesome enough with smaller structures, but when one data structure covers multiple banks, you have a "special" kind of ugly to deal with.

I am not an expert, but it looks like you have three choices. You can either constantly change banks and hope to keep up, try to calculate when the bank changes need to happen and just do them "on-demand", or use indirect addressing registers to get your data. (INDF0, etc.)

How well either of those translates to C, I cannot say.

There must have been some satisfaction in figuring out the bug though. That's a tough one.

I need to go find some breakfast.

Good Luck,

LyleHaze

Link to comment
Share on other sites

Thanks for the tips man.  actually that's what I thought, but it's almost all done using POSTDECx ... apparrently sdcc figured that much out  :D

Actually I have to say that so far I am rather impressed with SDCC's job. The ASM coming out of it seems really not too bad at all. I have noticed that with code where the results should be the same, the way you lay out the C code, can have quite a significant bearing on the output ASM... But once you've seen it done a few times it's not hard to get used to. It seems to like it if your code is broken down into constituents, kinda like writing ASM with ease, rather than writing in a way that takes full advantage of the relaxed methods that come with the higher level language that it is... C is interesting. ;D

I have a feeling that this is because I was forcing things where SDCC is supposed to automate them. I think I used too firm a hand  :-\ Once I just made room for it and stepped off, it seems to be OK. But I'm still uneasy....

Once I have the note offs working I'm gonna hardcode useful values into every step and clock, record the output and see if I get zeros... Which will be obvious because they'll cause the track to lose it and start spewing midi and crash java ;D

This is gonna take awhile :( At least it's a good excuse to do a demo ;)

Edit: There's this little voice in my head trying to remind me that one of the memory access methods, banked or indirect, is faster than the other.. any hints guys?

Link to comment
Share on other sites

C is interesting. ;D

Ancient Chinese Curse:  "May you live in interesting times"

A good friend, and a far better programmer than I will ever, be once called it a 'write only language'. He said he could write things in it well enough, but re-reading the code 6 months later he couldn't make sense of it.

On a more salient point, asking because I haven't gone to the mat with C on the MIDIbox yet, does the compiler support nested structures?

Mike

Link to comment
Share on other sites

Heheheh yet true. I am cursed with a shocking power of recall so I'm trying to make a habit of commenting all the way. It probably won't help ;)

Nested structs are all good :) In fact it's handling nested arrays of structs of nested arrays of structs nicely :)

It's tricky though.... 2.5.0 doesn't handle it sometimes, and 2.6.0 has a bug

The latest nightlies have given me this:

"input buffer overflow, can't enlarge buffer because scanner uses REJECT"

I'd recommend SDCC 2.6.0 build r4359, if I could find it... I'm kicking myself for not keeping a copy :(

Maybe I should report the bug? I don't think I could give them much more detail than that though...

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
 Share

×
×
  • Create New...