Page 1 of 2

NEStronic Game Music Synthesizer (a work in progress)

Posted: Sun Jan 21, 2018 12:47 am
by dkonigs
So there's this little project I've been working on for the past few weeks, which I think is at a good stage to start showing off. Its still very much a work-in-progress, but the most critical components are now working.

What I'm building is a contraption that can play NES game music using an authentic RP2A03 CPU/APU to do the audio synthesis. Beyond the RP2A03 itself, however, I'm attempting to do this with entirely modern components.

My system is basically composed of two halves:

The NES half contains the RP2A03, an SRAM chip, and EEPROM chip, an audio amplifier, and an I2C interface.

The "modern microcontroller" half currently contains an Espressif ESP32-WROOM and a microSD card interface. Eventually, I intend for it to have a display and some buttons for control. (I'm also seriously considering wrapping up a version of this project into an alarm clock, which uses the ESP32's Wi-Fi for network time sync.)

The system currently works by reading the music from VGM files, and generating a command stream to tell the 2A03 what to poke at its APU registers.

Here's a video of my first end-to-end test:
https://www.youtube.com/watch?v=97jic_WRrwY

And a follow-up playing a different track:
https://www.youtube.com/watch?v=eafaFr9Q_rU

And here's the work-in-progress schematic of the NES half of the system:
Image

At the outset of this project, the VGM format seemed like an ideal thing to start with. Lots of rips already available, and the format was literally "poke this value into that register, sleep for X samples, poke this value into that register, etc." In very little time, I was actually able to write a quick-and-dirty C program that could convert VGM into a dump of 6502 assembly (or data easy to iterate over in 6502 assembly). Of course it filled my memory to play only a few seconds of Super Mario Bros this way, but that was enough to give me a huge grin.

As I got deeper into it, I also learned about NSF. It seems like NSF has an even larger catalogue of rips and is more populate in the chiptune community. Unfortunately, NSF seems to only be viable on emulators (or if you have just the right combination of "architecturally correct NES /w whichever bank switching chip a particular game felt like using"). So when/if I decide to consider dealing with NSF files, I think I'm going to have to basically convert it to VGM first. (Either by hacking an emulator on my PC, or by running a stripped down emulator on the ESP32 that captures APU writes and sends them across.)

Anyway, enough rambling for now, and I'll save any involved back-stories and inspirations for a future blog post. I just felt like I had to get some sort of forum post about this project out there. :-)

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Sun Jan 21, 2018 1:29 am
by lidnariq
NSF always uses the same bankswitching hardware, or none at all. The problem for your current design is that the no-bankswitching-at-all model expects something resembling a 32K ROM in the upper half of the address space.

As far as VGM consuming too much memory, I'd think you should be able to build a message-passing loop between the 2A03 and the ESP32 ...

For whatever it's worth, a bare 2A03 doesn't have to worry about where it maps things: the bus conflict that would happen on reads from $4016 and $4017 won't happen (because that's external, between the 74'368 on the mainboard and whatever other hardware is present) OR on any conflict that would happen on reads from $4015 (because the NES hardware special cases that address and ignores the data bus on reads from that address).

(When the 2A03's TST pin 30 is high, reads from the entire range from $4000 through $401F ignore the external data bus entirely)

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Mon Jan 22, 2018 9:22 pm
by dkonigs
lidnariq wrote:As far as VGM consuming too much memory, I'd think you should be able to build a message-passing loop between the 2A03 and the ESP32 ...
I know its probably lost in the middle of my attempt at writing a "brief" post, but this is actually exactly what I did. I only tried cramming data into the EEPROM in my early "let's see if I can make sound" tests. My later tests (which are in those video links) use the ESP32 to read in the VGM file and generate a sequence of I2C commands that tell the 2A03 what to do. This works quite well, doesn't require much 6502 code, and is how I plan to make this all work.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Mon Jan 22, 2018 9:46 pm
by lidnariq
Glad to see that was the obvious thing to have done :)

I'd think you'd also be able to emulate the 6502 and NSF bankswitching hardware on the ESP32, and relay the audio writes to the physical 2A03... basically dynamically generating the VGM at play time. That's probably also what you meant ...

Something I'd think would be fun from a musical point of view, although not particularly useful from the point of view of replaying existing soundtracks, is using some kind of reconfigurable clock generator to choose what frequency the 2A03 runs at. Could be nice to get an extra octave off the bottom, or more precise tuning with higher notes, or different DPCM tuning. (Then again, most of the VGMs of Master System games I've listened to are dramatically improved by transposing everything down an octave. The 2A03 has a much less limited musical range, but I can't say that won't be true here too.)

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Tue Jan 23, 2018 10:20 am
by tepples
lidnariq wrote:Then again, most of the VGMs of Master System games I've listened to are dramatically improved by transposing everything down an octave.
Is it because their developers weren't aware of the alternate mode for pulse 3 that's 4 octaves down? That's two octaves below even the 2A03 triangle.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Jan 25, 2018 6:42 pm
by dkonigs
lidnariq wrote:Something I'd think would be fun from a musical point of view, although not particularly useful from the point of view of replaying existing soundtracks, is using some kind of reconfigurable clock generator to choose what frequency the 2A03 runs at.
This is an idea that has never occurred to me, but actually something worth considering as part of a follow-on project (in addition to exploring add-on audio chips only used on the Famicom). Programmable oscillators are now a thing (a Sega Genesis VGM player project used them in lieu of weird crystal frequencies) so its definitely do-able.

Thankfully I actually was able to still get a new crystal at the weird frequency used by the 2A03, otherwise I'd have already built in this capability.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Jan 25, 2018 9:21 pm
by tepples
So 6 * NTSC is "weird" now?

Is 4 * NTSC still gettable, and is PLL multiplication by 3/2 practical?

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Jan 25, 2018 9:36 pm
by lidnariq
6×NTSC crystals have been verging on "special order" territory for years now.

Not that special order is that much of a problem.

No PLL is needed; extracting the 3rd harmonic of a square wave using an adjustable LC tank is an easy thing.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Feb 22, 2018 12:19 am
by dkonigs
As I continue to work on this project, I decided to go ahead and start writing blog posts to describe it in more detail.

Here's the first post:
Introducing the Nestronic! (Part 1)

This blog post mostly covers what I already said in my first post on this forum thread, but goes into a lot more detail. (In the next post, I plan to cover the system architecture and schematics.)

As far as the actual project, the schematics are mostly finalized. I'm now neck-deep in PCB layout, and hope to get to the point of building some real (non-breadboard) hardware in the next month or so (time and resource permitting).

It feels pretty crazy that I do actually have this all assembled across three breadboards, but the general flakiness and noisiness of breadboard circuit connections is getting old.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Feb 22, 2018 1:10 am
by lidnariq
I didn't actually have problems with assembling my (8051-based) college final microcontroller project class on two of those breadboards ... but it was a brand-new one purchased just for the project.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Feb 22, 2018 9:15 am
by dkonigs
lidnariq wrote:I didn't actually have problems with assembling my (8051-based) college final microcontroller project class on two of those breadboards ... but it was a brand-new one purchased just for the project.
Most of these were brand-new too. :-)
Of course its often hard to determine which breadboards are "good" vs "crap" before you buy them. They all have certain issues beyond that, which may or may not manifest in your project.

The basics of this system actually worked just fine. I don't think I ever had a problem where the parts of the 2A03 "computer" failed to communicate with each other, or the ESP32 couldn't talk to some peripheral because of these breadboard issues.

The most annoying breadboard, mechanically-speaking, was the triple-wide one I moved the 2A03 system onto mid-way through the project. It took a lot more force and fiddling to get component pins into its holes, versus all the others. I think I have a lot of resistors with squiggly pins as a result of that.

From a flaky-connection perspective, I had the most issues with the connections between the audio output pins, amplifier, and speaker. It seemed like poking (or breathing) on various resistors and wires in that chain would determine whether or not I got good sound out of it.

Most of the other electrical issues came down to the effects of noise. I could detect the frequency of the crystal oscillator from all over the breadboard. However, I don't think this caused any serious issues. But what did cause sporadic issues, is the noise that appeared on the OUT ($4016) pins of the 2A03. The "low" value for these pins tended to float between ~600mV and ~800mV depending on when I measured it. When probed with an oscilloscope, I found ~300mV p/p noise on it. (Since I'm using those pins to control whether the amplifier is on or off, and the amplifier chip wants it <800mV for "off", this obviously caused issues.) Fingers crossed that this vanishes once I go to real PCB, otherwise I'll need to add a buffer between the two chips.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Feb 22, 2018 11:41 am
by lidnariq
The only really big difference I see between my board and yours is that I more aggressively made ground/power supply rails into a "plane", by connecting the +5V and ground rails at both top and bottom. (Yes that causes ground/power supply loops, no it seemed to have worked out) My design mixed baseband composite video on the same board as a bunch of discrete logic (implementing a genlocked framebuffer)

Looking at your board, power consumption in any of the left column will cause power supply sag as seen by the 2A03, and there's also no bypass capacitor for it.

But don't let me stop you from actually changing to a PCB.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Thu Feb 22, 2018 6:21 pm
by dkonigs
lidnariq wrote:The only really big difference I see between my board and yours is that I more aggressively made ground/power supply rails into a "plane", by connecting the +5V and ground rails at both top and bottom. (Yes that causes ground/power supply loops, no it seemed to have worked out) My design mixed baseband composite video on the same board as a bunch of discrete logic (implementing a genlocked framebuffer)

Looking at your board, power consumption in any of the left column will cause power supply sag as seen by the 2A03, and there's also no bypass capacitor for it.

But don't let me stop you from actually changing to a PCB.
Yeah, I have had issues with voltage drops across this breadboard setup. They were actually far more noticeable on the 3.3V rail the ESP32 is hanging off of, as they didn't actually cause problems on the 2A03's 5V rail. As its currently configured, the regulators have separate wires to each vertical track, which helps with it. I'll obviously do better on the PCB layout.

The bypass capacitors are missing on the breadboard because they were actually causing problems. When I had them installed, the crystal oscillator circuit sometimes didn't work correctly on startup. However, the actual schematic has bypass capacitors on all the ICs, so they'll definitely be on the real PCB.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Sun Mar 04, 2018 6:23 pm
by dkonigs
The next blog post is ready:

Nestronic System Architecture

This post details the architecture of the system, and shows all the circuit schematics.

As far as actual development goes, PCB layout is mostly complete and I've begin to work on the design of the enclosure. Once I have a bit more certainty on my reflow soldering assembly solution (long story), I'll actually order the PCB and components to start putting this thing together for real.

Re: NEStronic Game Music Synthesizer (a work in progress)

Posted: Wed Apr 04, 2018 3:17 pm
by dkonigs
Just finished getting the first prototype PCB assembled and mounted in a half-enclosure. Here's a video of it in action:
https://www.youtube.com/watch?v=izMFPKmD5ZU