# Synthetic Voice

> A voice made sample by sample, the way the machines of the eighties talked: it sings Daisy Bell, the first song a computer sang, and speaks Spanish read by rules.

Example `synthetic-voice`, running at https://nacatamalon.com/en/examples/synthetic-voice. The whole catalogue: https://nacatamalon.com/llms/examples.md.

`nacatamalon/speech` makes a voice sample by sample, the way the machines of the eighties talked: a throat that buzzes at a pitch, five formants that shape it into vowels, and noise for the s, the stops and the breath. In 1961 a computer sang *Daisy Bell* this way, the first song a computer sang.

The song's voice is a vocal part, `daisy-bell.vocal`, loaded with `useLoadVocal` and sung over the band with `accompany`: `useSong` plays them together, made as they play, so nothing waits to load. The same part is sung by another voice with a different `throat`, and `lyricsAt` gives the words as they are sung.

`renderSpeech` says text, and hands back a sound for `useSound`. Spanish is read by rules, and `textToPhonemes` shows how; anything else is written as phonemes. 1 to 4 play a take, space stops it.

## Code

`index.ts`:

```ts
import { createGame } from 'nacatamalon';

import * as Settings from './settings';
import { Voice } from './scenes/demo';

const game = createGame('#app', {
    width: Settings.WIDTH,
    height: Settings.HEIGHT,
    background: '#0a0a12',
});

// Declare Scenes
game({
    Voice,
});
```

`components/voiceplayer.ts`:

```ts
import { createText, useKeyboard, useSong, useSound, useUpdate } from 'nacatamalon';

import { accompany, lyricsAt, renderSpeech, textToPhonemes, useLoadVocal } from 'nacatamalon/speech';
import * as Settings from '../settings';
import { WALTZ } from '../data/waltz';

export const VoicePlayer = () => {
    const keys = useKeyboard();

    const daisy = useLoadVocal({
        src: Settings.DAISY
    });

    // Nothing is made before it plays: the songs are sung as they are heard, the speech made once here.
    const takes = [
        {
            label: 'DAISY BELL, WITH THE BAND',
            sound: useSong(accompany(daisy, WALTZ))
        },
        // The band silenced, and the voice as the part says but dead straight, with no breath.
        {
            label: 'DAISY BELL, 1961: ALONE, NO VIBRATO',
            sound: useSong(accompany(daisy, WALTZ, {
                singers: {
                    voice: {
                        transpose: -12,
                        vibrato: 0,
                        breath: 0
                    }
                }
            }), {
                voices: {
                    triangle: 0,
                    pulse1: 0,
                    pulse2: 0
                }
            })
        },
        // The same part, sung by a child: a shorter throat, an octave higher.
        {
            label: 'DAISY BELL, A CHILD WITH THE BAND',
            sound: useSong(accompany(daisy, WALTZ, {
                singers: {
                    voice: {
                        throat: 1.3,
                        transpose: 0
                    }
                }
            }))
        },
        {
            label: 'SPEAKING SPANISH',
            sound: useSound(renderSpeech({
                text: Settings.SPOKEN
            }))
        },
    ];

    createText({
        text: 'SYNTHETIC VOICE',
        style: {
            fontSize: 16
        },
        tint: Settings.WHITE,
        anchor: {
            x: 0.5,
            y: 0
        },
        transform: {
            x: 240,
            y: 30
        }
    });

    const rows = takes.map((take, i) =>
        createText({
            text: `${i + 1}  ${take.label}`,
            style: {
                fontSize: 8
            },
            tint: Settings.WHITE,
            transform: {
                x: 40,
                y: 80 + i * 24
            }
        }));

    // The words of the song being sung, as they are sung.
    const words = createText({
        text: '',
        style: {
            fontSize: 8
        },
        tint: Settings.ON,
        anchor: {
            x: 0.5,
            y: 0
        },
        transform: {
            x: 240,
            y: 190
        }
    });

    // How the rules read a word: what to copy and fix as phonemes when they read one wrong.
    createText({
        text: `HOLA: ${textToPhonemes('hola')}`,
        style: {
            fontSize: 8
        },
        tint: Settings.WHITE,
        anchor: {
            x: 0.5,
            y: 0
        },
        transform: {
            x: 240,
            y: 236
        }
    });
    createText({
        text: '1-4 PLAY   SPACE STOP',
        style: {
            fontSize: 8
        },
        tint: Settings.WHITE,
        anchor: {
            x: 0.5,
            y: 0
        },
        transform: {
            x: 240,
            y: 286
        }
    });

    let playing = -1;

    useUpdate(() => {
        takes.forEach((take, i) => {
            if (!keys.justPressed(String(i + 1))) {
                return;
            }

            takes[playing]?.sound.stop();
            take.sound.play();
            playing = i;
        });

        if (keys.justPressed(' ')) {
            takes[playing]?.sound.stop();
            playing = -1;
        }

        rows.forEach((row, i) => (row.tint = i === playing ? Settings.ON : Settings.WHITE));

        const take = takes[playing]?.sound;

        words.text = take !== undefined && 'beat' in take ? lyricsAt(daisy, take.beat)?.words ?? '' : '';
    });
};
```

`data/waltz.ts`:

```ts
import { type TSongDoc } from 'nacatamalon';
import { DOCUMENTS } from 'nacatamalon/files';

const CHORDS = 'F F F Bb F Gm C7 F F C7 C7 F F Gm C7 C7 C7 F C7 F F C7 Bb Gm F F C7 F C7 C7 F C7 F F'.split(' ');

const VOICINGS: Record<string, [string, string, string]> = {
    F: ['f,,', 'a,', 'c'],
    Bb: ['bb,,', 'bb,', 'd'],
    Gm: ['g,,', 'bb,', 'd'],
    C7: ['c,', 'bb,', 'e'],
};

const bars = (play: (chord: [string, string, string]) => string) => CHORDS.map((chord) => play(VOICINGS[chord]!)).join(' | ') + ' |';

export const WALTZ: TSongDoc = {
    format: DOCUMENTS.SONG.FORMAT,
    tempo: 180,
    meter: [3, 4],
    voices: {
        triangle: {
            notes: bars(([bass]) => `${bass}4 r r`),
            volume: 0.4
        },
        pulse2: {
            notes: bars(([, low]) => `r4 ${low} ${low}`),
            duty: 0.5,
            volume: 0.2
        },
        pulse1: {
            notes: bars(([, , high]) => `r4 ${high} ${high}`),
            duty: 0.25,
            volume: 0.2
        },
    },
};
```

`scenes/demo.ts`:

```ts
import { createScene, useSpawn } from 'nacatamalon';

import { VoicePlayer } from '../components/voiceplayer';

export const Voice = () => {
    useSpawn(VoicePlayer)();

    return createScene();
};
```

`settings.ts`:

```ts
import { getColor } from 'nacatamalon';

export const WIDTH = 480;
export const HEIGHT = 320;
export const DAISY = '/example-assets/vocals/daisy-bell.vocal';
export const SPOKEN = 'Hola. Soy la voz de Nacatamalón. ¿Me escuchas bien?';

export const WHITE = getColor('#ffffff');
export const ON = getColor('#f5c518');
```