Speakjet - A PIC ready sound chip?

[audiocommander]

Here's a raw demo video (a bit blurry, but better than nothing),

focusing mainly on the synth/envelope/subtractiveSynthesis features:

This is a technical demonstration of the kII SpeakJet Application v0.2.4

The harmonisation features' date=' base-note listening and scale settings are demonstrated at the beginning.

The first part shows synth sounds, with and without Enveloped OSCs and varies combinations of waveforms applied to a subtractive sound-synthesis together with waveforms of the envelope. I know the waveforms are a bit blurry, but one sees when it's changing (there are SINE, SAW, TRIANGLE, SQUARE).

In the late mid-part I am playing the HandTrackerMatrix (see http://www.youtube.com/watch?v=qSiMwp... ) on different channels. CHannels 11 to 16 are experimental and show a work-in-progress. The black bar at the top right shows the quantize value (1/4, 1/8, 1/16 or 1/32 notes).

The first part (synth) is played on a keyboard (microKONTROL), the second part (voice and experimental) on the handtracker sensor matrix.

[/quote']

Cheers,

Michael

ps: @RIO

if the handtracker was enabled, this sounds like unconnected, floating AINs :)

good.

[Rio]

ok.. OSCSynthesis with Sync? i'll check it out

... can you please record a mp3 from playing some notes with one of that OSCs without envelope activated. I wanna listen whether it is noiseless..

[audiocommander]

as already stated, sync (quantizing) is only enabled for the handtracker (AIN), 'cause it's no buffer-mechanism but a timed trigger-action. I got to do some urgent work now, I'll record the osc without env as soon as I find some free minutes...

In the meanwhile for all people trying to catch up all these posts lately, this is the important stuff:

- v0.2.4 beta

http://www.midibox.org/forum/index.php?topic=2870.msg63685#msg63685

- technical demo (synth sounds, quantising, harmonising, experimental handTracker modes)

regards,

Michael

[stryd_one]

Thats so darn cool it's crazy :D

[Wild_Weasel]

Seems like somebody did exactly the same:

http://www.getlofi.com/?p=775

... but I bet you can do it cheaper than 550$

Regards

Michael

[audiocommander]

yeah, this link came up somewhen before  :)

but thanks for your notification!

Though I'm especially proud of:

- MB Speakjet / kII is free!

- the current material costs are approx. 40 US$ to 100 US$, depending on which controls are used

- kII has a very crazy handTracker interface which behaves very different from the FLAME joystick. Whereas the joystick is triggering only the available sounds; kII has a scientific phoneme table with jaw and tongue controls. These controls are also available by Midi, so you don't necessarily need a handTracker device (or can attach something else!)

- kII has no sequencer; but as I'm a realtime-control freak: kII has an auto-synced quantizer

- and kII has an integrated harmonizer

btw

Here's another work-in-progress video:

I enhanced the Midi-Output and set up an Ableton-Live Set. Everything you see/hear is realtime interactive, no editing. Just one SpeakJet, three beat-samples (filtered, volumed) and two Operator-Instances (Note-On output by kII), harmonization and quantization from kII, clock from Ableton Live.

besides these enhancements, I'm already working on improved env/synth control on suggestions by Rio, overworked the kII services routine (timed phrase trigger, auto-cleanup). With the new timed event features, it is also very (very) likely that there will be real support for PIC internal phrases / recordings (not the ones inside the SJ)...  8)

... a bit of competition seems to be the best reason to keep things going, eh' ;D

Cheers,

Michael

[ultra]

is this thing (v.2) ready for me to build without many problems?  i have no interest in add-ons, just midi control, so that takes out some variables.  i'm just about completed with my midibox seq and looking for a new project.  this one looks great.  do you consider the 1x8 lcd to be the final design?  i saw it started with a 2x16 or something like that.

[audiocommander]

Hi ultra,

if you're just interested in midi-control of the sj, it's indeed ready to be used and quite future-proof. It's very unlikely that the hardware will change; there will just be some more software-features added, if there is enough interest.

This is the latest version: ?topic=2870.msg63685#msg63685 date=1176227557

The project was not very active the last months because the current status has been sufficient to me and there has been no one else that built this. So if there is interest, I will surely resume to work on this project.

---

these things are still to be done:

- a base-note bug (fixed this in the sensorizer but not yet in the kII, just a matter of time)

- software phrase support (not that easy due to possible buffer overflows, critical timing issues; would do that if there is enough interest)

- hui input (buttons, encoders...) I have not done this, because the external phrase support and functionality is not thought over. Also, I'm not convinced that a (computer) keyboard is a wise input device, because when it comes to speech, we're not talking about words and chars, we're dealing with phonemes. Introducing typed chars requires a word-to-phoneme dictionary that cannot ever be complete and is restricted to the english language. Moreover (and this is a personal believe), I like the idea of not just imitating the human language, but to take the electronic speech one step further in musical terms.

these things won't be done by me, though it would be okay for me to integrate them if they are coded well and easy to take over (namespaces, blackboxed/encapsuled functions):

- extra manual envelopes like suggested by rio. In my eyes, these just produce unnecessary extra serial messages, that make it more probable to overflow the 64 byte input buffer. I think it would be smarter to use a better suited chip (like the soundGin) for things like these, it has by far the better synth capabilities.

- phrase storage write access to the SJ. this is a critical issue, because if there is some small error in the buffer (-> again my friend the overflow) it might write bad data to the chip and possibly destroy the firmware. Anyway: regarding the low storage space (IIRC just 256 phonemes for all 16 phrases), I guess it's a lot easier to use the provided magnivation PC-program to save some default phrases - and instead add an external phrase manager. this would have the advantage to enable the storage of lots of phrases in a bankstick module.

---

I am using a 1x8 lcd because I think that's sufficient (harmony/channel/envelope/osc-env/beat-quantisation). If you have trouble finding 1x8 lcds, I can implement support for 2x16 of course, too (I got some 1x8 extremely cheap from pollin)

please ask if you have more question; though I will be on holidays the next days, so do not wonder if my answers will take a bit longer ;)

Cheers,

Michael

[ultra]

audiocommander,

thanks for your response.  i think i'll go ahead and start to build this project.  i am interested in it just to mess around with the speech sounds and not to have it make words or sentences.  more or less, i'm always on the lookout for unique sound generators and this seems to fit the bill.

if i understand correctly, the end result will be a simple box with midi input and audio output that is completely controlled via midi?  if that's the case, will i have to modify the code to receive whatever midi cc #'s my controller sends out?  i am not wanting to re-map any of my midi controllers and i'm able to modify the code for cc#'s if it's a relatively simple task.

last thing is, i have built a midibox seq (well, nearly completed) and a number of other small electronics projects.  that is all the experience i have.  i do have a pic programmer coming in the mail tho.  will it be bothersome to you if i have a number of questions related to building this project?  of course, i will use documentation as much as possible but i will no doubt have questions, and you seem to be the only one who can answer them.

thanks!

drew

[audiocommander]

more or less, i'm always on the lookout for unique sound generators and this seems to fit the bill.

if i understand correctly, the end result will be a simple box with midi input and audio output that is completely controlled via midi?

yep :)

it's truly unique, but beware though that the sound is of course not high quality (there's a bit noise on the output, but it's okay for me)

will i have to modify the code to receive whatever midi cc #'s my controller sends out? i am not wanting to re-map any of my midi controllers and i'm able to modify the code for cc#'s if it's a relatively simple task.

that's quite easy' date=' you just have to change the defaults for all SJCC_*** in IIC_SpeakJetMidiDefines.h

...maybe in ACMidiProtocol.h too, where harmony changes are defined, but I found that useful to have a standard for inter-device and -application harmony changes; for example: if you should ever build an ACSensorizer, the kII automatically changes it's current harmony (base+scale) if the Sensorizer does and vice versa.

/*
 *  IIC_SpeakJetMidiDefines.h
 *  kII
 *
 *  Created by Michael Markert, audiocommander.de on 26.08.06
 *  Copyright 2006 Michael Markert, http://www.audiocommander.de
 *
 */

/*
 * Released under GNU General Public License
 * http://www.gnu.org/licenses/gpl.html
 * 
 * This program is free software; you can redistribute it and/or modify it under the terms
 * of the GNU General Public License as published by the Free Software Foundation
 *
 * YOU ARE ALLOWED TO COPY AND CHANGE 
 * BUT YOU MUST RELEASE THE SOURCE TOO (UNDER GNU GPL) IF YOU RELEASE YOUR PRODUCT 
 * YOU ARE NOT ALLOWED TO USE IT WITHIN PROPRIETARY CLOSED-SOURCE PROJECTS
 */


#ifndef _IIC_SPEAKJETMIDIDEFINES_H
#define _IIC_SPEAKJETMIDIDEFINES_H


// include standard MIDI names
// you can then use MIDI_CC_VOLUME instead of 0x7
#include "ACMidiDefines.h"


// ********* SPEAKJET MIDI ASSIGNMENTS * //
// MIDI ASSIGNMENTS
// optimized for KORG microKONTROL
//
// just assign the ControlChange or Channel Numbers to the functions
// eg: CC 107 on CH 1 controls the SPEED of the SpeakJet Allophones (SJCC_SPEED)
// eg: Notes  on CH 2 trigger allophones (all phonemes) (SJCH_ALLOPHONES)
//
// some midi-controls are hardcoded (can't be assigned here)
// channel control messages like PITCH-WHEEL, AFTERTOUCH or PANIC and
// system realtime messages like START/STOP/CONTINUE




// ********** CHANNELS **********
// channel assignments for NOTE_ONs 0x90 and NOTE_OFFs 0x80 functions
// MIDI CHANNELS for NOTE_ONs
#define SJCH_SOUNDCODES         1   //  MSA Soundcodes (Allophones + FX)                        default 1
#define SJCH_ALLOPHONES         2   //  MSA Soundcodes (Allophones)                             default 2
#define SJCH_FX                 3   //  MSA Soundcodes (FX)                                     default 3
#define SJCH_PITCH              4   //  MSA Pitch (Note2Freq) only! No sound output!            default 4
#define SJCH_VOWELS_DIPHTONGS   5   //  MSA Pitched Vowels incl. Diphtongs                      default 5
#define SJCH_CONSONANTS         6   //  MSA Pitched Consonants  (depending on jaw & tongue)*    default 6
#define SJCH_VOWELS_CONSONANTS  7   //  MSA Pitched Vowels & Consonants (mixed)                 default 7
#define SJCH_CONSONANTS_OPEN    8   //  MSA Pitched Consonants produced by mouth opening        default 8
#define SJCH_CONSONANTS_CLOSE   9   //  MSA Pitched Consonants produced by mouth closing        default 9
#define SJCH_PERCUSSIVE         10  //  MSA Soundcodes (Percussive Consonants)                  default 10
#define SJCH_OSC1               11  //  OSC 1 (Note2Freq, monophon)                             default 11
#define SJCH_OSC2               12  //  OSC 2 (Note2Freq, monophon)                             default 12
#define SJCH_OSC3               13  //  OSC 3 (Note2Freq, monophon)                             default 13
#define SJCH_OSC4               14  //  OSC 4 (Note2Freq, monophon)                             default 14
#define SJCH_OSC5               15  //  OSC 5 (Note2Freq, monophon)                             default 15
#define SJCH_OSC_SYN            16  //  OSCs 1 to 5 (Note2Freq, harmonic polymode)              default 16 

// the following channel types can also be used, but make sure only 16 channel nums are defined!
#define SJCH_VOWELS             255 //  MSA Pitched Vowels only (depending on jaw & tongue)*    default 0 (n/a)
#define SJCH_DIPHTONGS          254 //  MSA Pitched Diphtongs only (depending on jaw & tongue)* default 0 (n/a)



// ********** CONTROL CHANGES **********
// general
#define SJCC_CHANNEL_ASSIGNMENT_TOGGLE  17  //  switch to next channel assignment, CH1=>CH12 (channelAssignment, main.h)
#define SJCC_CHANNEL_ASSIGNMENT         18  //  set channel assignment, 0..127 => 1..16 (channelAssignment, main.h)
#define SJCC_PHRASE             9

// pauses: pause 0(0ms) / 1(100ms) / 2(200ms) / 3(700ms)
//         pause 4(30ms)/ 5(60ms)  / 6(90ms)
#define SJCC_PAUSE0             255 //   0 ms   // inactive!
#define SJCC_PAUSE1             22  // 100 ms
#define SJCC_PAUSE2             23  // 200 ms
#define SJCC_PAUSE3             24  // 700 ms
#define SJCC_PAUSE4             254 //  30 ms   // inactive!
#define SJCC_PAUSE5             21  //  60 ms
#define SJCC_PAUSE6             253 //  90 ms   // inactive!
// modifiers: slow/low/high/fast
#define SJCC_NEXT_SLOW          25  // play next phrase ...
#define SJCC_NEXT_LOW           26
#define SJCC_NEXT_HIGH          27
#define SJCC_NEXT_FAST          28
// phrases: call phrase
#define SJCC_PHRASE0            29  // play internal phrase ...
#define SJCC_PHRASE1            30
#define SJCC_PHRASE2            31
#define SJCC_PHRASE3            32
// step phrase record (not activated by default!)
#define SJCC_PHRASE_REC_UNDO    33  // delete last recorded command
#define SJCC_PHRASE_REC_ABORT   34  // stop recording, delete buffer
#define SJCC_PHRASE_REC         35  // start recording
#define SJCC_PHRASE_REC_PREVIEW 36  // preview buffer
// harmony control (also see ACMidiProtocol.h!)
#define SJCC_HARMONY_LISTEN     37  // 64-127: listen for next note-on, 0-64: exit listen-mode
#define SJCC_HARMONY_SCALE      38  // 0-SCALE_MAX: set scale, 127: next scale
#define SJCC_HARMONY_BASE       39  // set new base note
// articulation controls
#define SJCC_MOUTH_JAW          40  // jaw position (low/mid/high)
#define SJCC_MOUTH_TONGUE       41  // tongue position (front/center/back)
#define SJCC_MOUTH_HEIGHT       42  // pitch, note-height (send only! use notes on  CH# SJCH_PITCH to receive!)
#define SJCC_MOUTH_STRESS       43  // same as bend
#define SJCC_MOUTH_SPEED        44  // same as speed
#define SJCC_MOUTH_PAUSES       45  // inserts pauses (long/short) 
#define SJCC_MOUTH_PHONEME      46  // last phoneme (send only! use notes on CH# SJCH_ALLOPHONES to receive!)
// OSCx Frequency
#define SJCC_OSC1_FREQ          101
#define SJCC_OSC2_FREQ          102
#define SJCC_OSC3_FREQ          103
#define SJCC_OSC4_FREQ          104
#define SJCC_OSC5_FREQ          105
// OSCx Level
#define SJCC_OSC1_LVL           111
#define SJCC_OSC2_LVL           112
#define SJCC_OSC3_LVL           113
#define SJCC_OSC4_LVL           114
#define SJCC_OSC5_LVL           115
// OSC controls
#define SJCC_OSC_WAVESHAPE      50  // switch to next osc additive synth waveshape
#define SJCC_DISTORTION         118 // distortion level (mix) for OSCs 4 & 5
// ENV controls
#define SJCC_ENV_OSC_A          53  // ENV on/off for OSCs 1 to 3
#define SJCC_ENV_OSC_B          54  // ENV on/off for OSCs 4 and 5 (half enveloped only)
#define SJCC_ENV_OSC_TGL        52  // toggle ENV on/off for OSC groups (00, 01, 10, 11)
#define SJCC_ENV_WAVESHAPE      116 // set ENV Waveshape 1:0..31, 2:32..63, 3:64..95, 4:96..127
#define SJCC_ENV_WAVESHAPE_TGL  51  // switch to next envelope waveshape, also see SJCC_ENV_TYPE
#define SJCC_ENV_FREQ           106
#define SJCC_SYNTH_ENV          56  // ENV on/off for Additive Synthesis
// speech params
#define SJCC_SPEED              107 // same as SJCC_MOUTH_SPEED
#define SJCC_BEND               117
#define SJCC_MASTER_VOL         108

// Joystick X/Y: Pitch / Env Freq
#define SJCC_PITCH              MIDI_PITCH
#define SJCC_FREQ               MIDI_POLY_AFTER     // same as bend
[/code]

maybe I need to add some words about the Channel modes (SJCH_***):

the channels at the top of this file are used to play in different modes; eg. when you want to trigger consonants only, you can send notes from CH6. But there is also a channelAssignment; if this is set to a value between 1 and 16, you can change the "default" behavior, eg. if you send notes from CH6 but the ChannelAssignment is set to 3, you trigger FX sounds only. This "mode"-number is also shown in the 1x8 LCD.

And you may need to set KII_AIN_ENABLED in main.h to 0 - I will make this the default setting in the next update anyway; I can release this as soon as I'm back.

If you have troubles in recompiling the sources, I can also do this for you and send you a .syx file if you submit me your adapted IIC_SeakJetMidiDefines.h.

that is all the experience i have. i do have a pic programmer coming in the mail tho. will it be bothersome to you if i have a number of questions related to building this project? of course, i will use documentation as much as possible but i will no doubt have questions, and you seem to be the only one who can answer them.

no problem, ultra :)

just don't forget that I'm away the next days.

I hope that assembling the IIC_Speakjet after this tutorial shouldn't be harder than assembling a Core Module. Just follow the PDF (the different views should help keeping the overview).

burning the firmware to the PIC16F shouldn't be big of a problem either. If you have troubles, I'm sure someone will burn the chip for you (I could do this as well).

Best,

Michael

[ultra]

thanks a lot.  on the hardware side of things, i should be ok to build it.  the software i might need some help with, but i have written code before so at least i can recognize what's going on.  it will be a while until i can start anyway as i have to put in a parts order.  the fact that this responds to different midi channels in different ways could make it an interesting companion to the magic midi delay, which can route delays to other channels.  :)

i'll keep you updated on what i come up with.

[ultra]

hey do you have a part number for that crystal?  preferably one that will show up on mouser.  there's so many options when i type in "20 mhz crystal" and i don't know what to get.  thanks

[audiocommander]

wow, this is what I call a nice navigation  :-X

I dunno, have no part number, but basically it's the same as used on the Core Module, just with 20 MHz instead of 10 MHz: http://www.mouser.com/search/productdetail.aspx?R=XT9S20ANA10Mvirtualkey61300000virtualkey73-XT49S1000-20

http://www.midibox.org/dokuwiki/parts_faq#what_type_of_crystal_should_i_buy

Best,

Michael

[th0mas]

Somewhat OT:

Anyone think that the speakjet's what Boys Noize uses?

http://www.myspace.com/boysnoizemusic, not sure what tracks he has on his myspace (and can't check, no flash at work), but most of them feature the robot sounds.  & down, for example.

[audiocommander]

hmm, can't see nothing there (no entries, no tracks, no vids).

[th0mas]

oops! Sorry, shouldn't be trying to do this at work..

http://www.myspace.com/boysnoizemusic

After listening some more I'm almost positive that a track on his album uses the speakjet, listen here:

http://www.woopwoopwoop.com/boysnoize-arcade_robot.mp3 (its a promo so I don't think I'll get in trouble :) )

However I think he might also have some other way of doing a more hi-fi robot voice (like the promo voice that repeats throughout the track), possibly microsoft SAM -> bitcrusher or something similiar.

[audiocommander]

no speakJet, these are all Mac OS X voices  ;)

Fred, Junior & Victoria

but interesting nevertheless!

ps: for all mac users select this text and choose > Services > Speech > Start Reading Text:

[[inpt TUNE]]

~

2EH {D 4000; P 400.0:0 400.0:40 200.0:60 200:100}

[[inpt TEXT]]

nice, isn't it ;D

Cheers,

Michael

[th0mas]

hahaha!  Why didn't I think of that? My gf and I were playing with that on my mac a couple weeks ago :D

Are you sure the mp3 doesn't have the speakjet though? it has a very different robot voice at the start counting down. 

Also: do you have a link on the syntax of that mac voice commands?  I only knew the "say" commandline command.

[audiocommander]

oh yeah, the countdown... haven't heard that in the first run... not sure what that is, I doubt it's a speakjet, but you're right it's quite close.

Here's the doku of the speech commands:

http://developer.apple.com/documentation/UserExperience/Conceptual/

SpeechSynthesisProgrammingGuide/FineTuning/chapter_4_section_3.html

The Tune-Format is quite interesting. There's also a very, very, very beautiful application shipped along with the Developer Tools called "Repeat After Me" with which one can easily generate TUNE formats (HOT Tip!!)

Cheers,

Michael

[th0mas]

Thanks for those references!! Time to go master that app :D

[Rio]

that countdown sample sounds like: SP0256-AL2

(That ICs are in special expansion moduls for C64 or Zx Spectrum)

[ultra]

hi.

i have the speakjet board etched and assembled, and so far i have a couple questions.  is j7 necessary for anything?  it says it's for the event input and i don't know if that's something i'll be using.  where does it connect to?  or is this an alternative jack to control the speakjet in other ways?  also, where is R4 (50k).  based on it's value i assume it's a pot.  volume?  where do i connect it?

what connects to pins 4 and 6 on the rs232 jack?  i'm just wondering because they're colored differently than the unused pins.  and off the board, there's a brown colored line but where does that go?

thanks!

[audiocommander]

Hi ultra,

the J7 pins have to be grounded if not connected. It's all described very well in the datasheet, please have a look. Basically these are I/O pins that either indicate wheather or not something is being spoken or you can use them as digital inputs to control phrase triggering. You might even hook up a RC remote!

I haven't used them so far, but it would be a lot harder to add connections if I need them one day, than to just set a jumper to ground an unneeded connection, so it does not hurt to have them...

Pins 4 and 6 of the RS232 have to be connected with each other (as the color indicates).

The brown RS232 line of the board remains unconnected; it's just brown because I used a colored 5-pin ribbon cable with these colors ;)

R4 was indeed a volume pot of the last version, but I removed it because it was quite useless (the SJ is not loud enough anyway) ...forgot to delete it from the list ;D

Best,

Michael

[ultra]

thanks audiocommander.

i have sound from my mb speakjet now but all i can get is it repeatedly saying "ready" when i start it up, and after a while it changes to some distorted bell type sound, then back to "ready".  pressing keys on my keyboard isn't doing anything, and i've also tried to assign knobs on my kb to what you have listed on the midi implementation and i get nothing.  it looks like your midi implementation is for the last version (i installed the latest), but i would think the keys should give me something regardless.  any ideas?

thanks,

ultra

edit:

it occurred to me while i was working this morning that i didn't solder in the cap on the back of the board.  i'm also confused about a wire on the back that's shown in your pictures.  not the one that connects "8" to "8" but the other one.  is that from the old version or something i need to worry about?

[audiocommander]

Hi Ultra,

the normal behaviour would be that it says 1x "READY" once it's powered on and then it awaits your commands.

Which kII-version are you using?

is it 0.1 or 0.2?

If it's 0.2: Have you grounded your AIN-Inputs? If not, I'm not sure wheather you use a beta-version with the HandTracker enabled or not. The easiest way to make sure it has nothing to do with the HandTracker is to clamp your AINs to ground. This is not needed for 0.1.

(I will fix this with the next update, but haven't time for this until next week).

If it keeps saying ready, this means most probably that the SJ is restarting over and over again - which is most likely a result of a short somewhere or a wrong connection from/to Pin 11 of the SJ (Restart).

Is the PIC16 LED on?

Don't let your SJ chip be powered on too long before you solved this. It's clear that nothing else is working, when the chip isn't yet connected properly.

You should definitely check your board all over again and look for shorts or wrong connections.

Does the main app (kII / Core Module) keeps restarting too? (Or other question: are you sure your Core module is okay?)

You could also try to power the SJ-board alone and connect it to your PC to see if the serial connections works (I doubt it).

The cap on the backside is just to improve the power; it shouldn't have no huge effect if you haven't soldered it.

The "other cable" on my photographies goes from PIC16:8 to MAX:12; my board was too small and this way I could use it ;)

Best,

Michael