Nessy: Play NES as a MIDI synth with USB CopyNES Installed

Discuss NSF files, FamiTracker, MML tools, or anything else related to NES music.

Moderator: Moderators

Post Reply
User avatar
orchetect
Posts: 2
Joined: Mon Sep 03, 2018 12:25 pm
Location: BC, Canada
Contact:

Nessy: Play NES as a MIDI synth with USB CopyNES Installed

Post by orchetect »

Thought I'd share this here. After modding a NES console with a USB CopyNES, I've written a desktop-side MIDI driver that pushes live APU register changes (in real-time!) to the NES with zero latency. The connection speed is fast enough to stream LFO data streams (ie: vibrato, tremolo) and even DMC PCM samples in real-time without the need for storing them in NES CPU memory and triggering. Which opens up a ton of possibilities.

In a nutshell: I wrote a custom desktop MIDI driver that holds all 2A03 register state information and contains all the logic to translate MIDI notes, CCs, etc. to appropriate APU register changes ($4000 ... $4017). The only data being transmitted over USB (currently 460800 baud rate but higher rates may be stable) is the actual register bytes being received on an interrupt in real-time, so it's essentially zero-latency and very responsive. (It's utilizing a USB CopyNES installed in the system as a way to write to the APU registers.)

The goal is to make all features of the 2A03 chip available as MIDI CCs so you can emulate the behavior of trackers in a MIDI-compatible DAW (Logic, Cubase, Pro Tools, Reaper, Ableton Live, etc.) and to aim it towards power-users (surpass the basic feature set of the midiNES / Chip Maestro) and thereby make it available for sequencing in parallel with other synths/samplers and audio in your DAW and not be restricted to working within a tracker.

Current features of note:
- implemented software-driven modulation (vibrato, tremolo, etc. - any APU parameter can be modulated with depth & rate)
- implemented software-driven sequence macros (akin to Plogue Chipsounds' Wave Sequencer) so note triggers can execute complex sets of user-programmed APU events during the attack, hold (loop), and release period of notes. These presets can either be stored in your DAW session as MIDI SysEx data so they are portable and platform-agnostic, or stored in a global settings file that the driver can load from disk when you open your DAW session

In future versions it might be possible to:

- stream NSF files direct from the desktop in real-time without requiring a RAM cart or side-loading/buffering
- have potential to receive real-time direct APU output stream from Famitracker or a modded NES emulator

Here's some preliminary videos demonstrating that it works. I'll get a full video put together eventually when all the features are implemented.

https://twitter.com/orchetect/status/10 ... 1755319296

https://www.instagram.com/p/BpEBwqAADaK

:beer: :D
Last edited by orchetect on Sun Dec 02, 2018 9:14 pm, edited 5 times in total.
Rahsennor
Posts: 479
Joined: Thu Aug 20, 2015 3:09 am

Re: Nessy: A live playable NES as a MIDI synth over USB

Post by Rahsennor »

Would you believe it, just yesterday I thought "wouldn't it be cool if you could hook a MIDI keyboard up to a NES and play chiptune music live?" Awesome work. :beer: :D

(notices username)

...heh. Small world. Welcome to NesDev!
User avatar
Memblers
Site Admin
Posts: 4044
Joined: Mon Sep 20, 2004 6:04 am
Location: Indianapolis
Contact:

Re: Nessy: A live playable NES as a MIDI synth over USB

Post by Memblers »

Very cool to see something like this coming together! I've worked on some very similar stuff over the years. Register dumps do work really well over USB/RS232. By the time I had my whole MIDI control and channel-stealing priority system set up, I ended up having a lot of fun making my own soft synth with wavetable, phase distortion, etc, making a General MIDI instrument set and all that, I got way sidetracked from outputting it to NES.

The DPCM channel poses some challenges when using the controller port, but it can be done. If you're doing asynchronous communication, DPCM steals CPU cycles, if you're doing synchronous, DPCM inserts some spurious clocks. blargg had shown that async NSF streaming with DPCM can be done though. And I've been working on a synchronous USB adapter that should be able to filter out the bad clocks.

Flash memory would work about as well as RAM. If you wanted an existing board, there is GTROM which could hold as much as 256kB of DPCM samples.
User avatar
Jedi QuestMaster
Posts: 688
Joined: Thu Sep 07, 2006 1:08 pm
Location: United States
Contact:

Re: Nessy: A live playable NES as a MIDI synth over USB

Post by Jedi QuestMaster »

This looks very interesting! I always wanted a MIDI -> NES device but was always hesitant to buy midiNES / maestro do to their limitations.

Would it be feasible to have it play arps when 3 keys are hit at the same time?
Post Reply