Miracle Piano in FCEUX

Discuss emulation of the Nintendo Entertainment System and Famicom.

Moderator: Moderators

GreyRogue
Posts: 51
Joined: Fri Dec 08, 2017 5:12 pm

Miracle Piano in FCEUX

Post by GreyRogue »

Edit--Missed a file (git.h)
TL;DR
Find the code for the Miracle Piano for FCEUX at the bottom of this post.

TS;GMUD (Too Short; Give Me Unnecessary Details)
So, a year and a half ago, I was watching TMR (The Mexican Runner) playing the Miracle Piano as part of his goal to beat all the NES games. He apparently had two sections (07.05 and 26.07) that he couldn't complete. I thought I could try and emulate it and figure out why.
On and off for the past year and a half, I've been kicking around with the code to get it work, and tried several different MIDI APIs to do it, including FluidSynth, RTMidi, and WINMM. I'd make progress with each one and then lose interest. A couple of weeks ago, I found out that MESS/MAME had already done it.
So, long story short, I just copied the PortMIDI they were using, and then just copied in the files that were using it into FCEUX. It didn't take too much modifying to get it work.
I've successfully tested it with an actual Miracle Piano connected with a cheap USB MIDI cable (IN=OUT, OUT=IN based on my labels), a Yamaha keyboard with the same cable, and software only MIDI using Virtual Midi Piano Keyboard(VMPK) through MIDI Yoke and MIDI-OX. All three work as expected. Most of the sound (keys pressed and accompaniment/sound effects) comes from the keyboard, so make sure you route the output midi to somewhere if you want to hear anything but the metronome. I routed both the input and the output to the Windows GS Wavetable Synth to hear it when using software MIDI.
There are some caveats. The Miracle Piano doesn't use the standard instrument numbers, so when it tries to set these, they may not match (for example if the game were to try to set the instrument to a pipe organ, most keyboards would set up for a harpsichord. Sound effects similarly won't match.
This could be worked around by using a translation matrix that intercepts these commands and converts them accordingly. I didn't implement this.
I was able to beat 07.06 and 26.08 (and 07.05 and 26.07 - see below) with the Miracle Piano through the USB-Midi cable, so it works well enough to actually play.

With a working debugger, I found that 07.05 does have issues. If you play frame perfectly how it's asking you to, you will fail. Every time it changes the page, it loses time about 1/10 of a note's worth of time (6 frames). In other words, after five pages, notes on the keyboard are received half a note late to the game (about half a second). I wrote a lua script that shows timing that works. Just hit the notes when the carrot moves horizontally. You can block the distracting built-in timer by uncommenting the lines in the script.
I then hooked my Miracle Piano upped to my actual NES and played it while watching the screen on my emulator. I had to press 'A' to start the emulator a couple frames earlier than on the NES to get them to line up (a little bit of lag there-you can tell they're lined up if the emulator screen and the NES TV both flash at the same time when changing pages), but I managed to complete the section successfully on the actual NES by using the emulator's LUA script timing.
The issue with 26.07 is different. 26.08 doesn't have this issue, but on 26.07 the tied notes are not working correctly. If you play them as indicated, you will fail. Instead you should just ignore the tied half notes.
--Edit--
Clarification: When you come to a series of tied notes, play the first one as written, but then treat all the additional tied notes as if they aren't there. i.e. Release when the first note would end if it didn't have any tied notes. The additional notes are not played in any way.

This will let you pass. I took what I learned from this and managed to beat all 3 sections of 26.07 (it speeds up after each section to eventually match the speed of 26.08) on the actual NES.
I did manage to beat both 7.05 and 26.07 (all three stages of 26.07) on the emulator. There may been some minor audio/video lag, which will affect how you need to play, but I made it through eventually.

If you want to see my data/notes:
https://docs.google.com/spreadsheets/d/ ... aFyPk0QhvI

These changes are in very rough shape. I got them to work and then stopped. They need serious clean-up/improvement (or even starting over and just using this a reference) before they're ready for merging. The PortMIDI ID numbers for in and out are hard coded (there should be a configure button), and it always attempts to connect. The PortMIDI code is not as isolated as it should be from the input device code like others are. I made the bare minimum changes to the files from MAME to get it work. If anyone else wants to work on it, feel free; it did what I needed, so I'm not planning on improving it.


All the hard work for Miracle Piano was done by the MAME people (R. Belmont and Fabio Priuli I believe). Thanks go to them.
Also, the specs from Nocash were invaluable.

To use these, copy the PortMIDI files from the MAME code into FCEUX/src/drivers/win/portmidi. Build it there (at least the static build).
Then copy miraclepiano.cpp into FCEUX/src/input.
Copy git.h and the correct input.cpp into FCEUX/src.
And copy these files into FCEUX/src/drivers/win:
Midi.cpp/h, input.cpp, miracle.cpp/h, portmidi.cpp, portmidiheader.h
miracle.cpp/h and portmidi.cpp and portmidiheader.h are MAME files that have been minimally edited.

Miracle Piano is connected to Port 1 and a controller is used to control the game from Port 2.
Attachments
MiracleMod_v2.zip
(44.25 KiB) Downloaded 479 times
Last edited by GreyRogue on Sun Dec 27, 2020 3:58 pm, edited 2 times in total.
User avatar
thefox
Posts: 3134
Joined: Mon Jan 03, 2005 10:36 am
Location: 🇫🇮
Contact:

Re: Miracle Piano in FCEUX

Post by thefox »

Pretty cool! I was also watching TMR play this game.

Ridiculous that there are bugs in some of the sheets...
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: fo.aspekt.fi
Dugongue
Posts: 5
Joined: Wed Apr 18, 2018 4:53 pm

Re: Miracle Piano in FCEUX

Post by Dugongue »

Hey thanks for letting me know about this. I'm going to try and go back to the lessons I couldn't finish now.
I've seen some other minor bugs like notes being transcribed incorrectly in certain sections too.
I also own the SNES cable so I might see if these bugs exist on that version too, or even the DOS version using DOSBox (haven't tried to get this to work yet).
DevanWolf
Posts: 2
Joined: Mon Sep 03, 2018 10:26 am

Re: Miracle Piano in FCEUX

Post by DevanWolf »

Can someone compile this into the standalone FCEUX source code?
User avatar
DarkCecil
Posts: 5
Joined: Tue Dec 22, 2020 8:24 am

Re: Miracle Piano in FCEUX

Post by DarkCecil »

I hate to keep necroing this thread, but I don't really see any mentions of Miracle Piano on any other NES emulator.

I am attempting to compile FCEUX with PortMIDI + MiracleMod files included. Every time I try, I get these build errors:

1> Creating library ..\output\fceux.lib and object ..\output\fceux.exp
1>input.obj : error LNK2001: unresolved external symbol "int __cdecl MidiInitialize(void)" (?MidiInitialize@@YAHXZ)
1>input.obj : error LNK2001: unresolved external symbol "void __cdecl MidiUpdateState(void)" (?MidiUpdateState@@YAXXZ)
1>input.obj : error LNK2001: unresolved external symbol "void __cdecl MidiClose(void)" (?MidiClose@@YAXXZ)
1>input.obj : error LNK2001: unresolved external symbol "struct INPUTC * __cdecl FCEU_InitMiraclePiano(int)" (?FCEU_InitMiraclePiano@@YAPAUINPUTC@@H@Z)
1>..\output\fceux.exe : fatal error LNK1120: 4 unresolved externals
1>
1>Build FAILED.

I'm using fceux-2.2.3 as my source, and the PortMIDI files from the most recent mame-master source. I'm building both of them with Visual Studio 2010. Unmodified FCEUX compiles with no issues whatsoever.

I'm not much of a developer, but I made it this far. Any ideas as to what's causing this to happen?

Thanks
User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Miracle Piano in FCEUX

Post by Dwedit »

Sounds like a file is missing from the project and needs to be added in?
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
GreyRogue
Posts: 51
Joined: Fri Dec 08, 2017 5:12 pm

Re: Miracle Piano in FCEUX

Post by GreyRogue »

New files need to be added to the solution.
e.g. The first three functions are in Midi.cpp. It won't build unless you add that file to the solution.
I can probably go dig up my vcxproj files if you can't get them to work right, but it shouldn't be too hard to add the files yourself. Let me know if you can't get it.
User avatar
DarkCecil
Posts: 5
Joined: Tue Dec 22, 2020 8:24 am

Re: Miracle Piano in FCEUX

Post by DarkCecil »

I actually just did that :) Thanks very much for getting back.
I was able to get it to build completely by adding the same files from MiracleMod to the solution file as well.

Now, I don't own an actual Miracle Piano, so this very well could be the cause of my next problem. I was trying to use my Novation SL-Mk2 via USB. But when I go to run the completed build, it crashes at the MidiInitialize() function of Midi.cpp, on line 424. If I skip past that to line 432 ( sleep(10) ), it will recover and I can use FCEUX again.

However, upon configuring P1 = Miracle Piano/P2 = Gamepad and loading the rom, it does not detect any inputs from my keyboard, only the gamepad. But since I don't own a Miracle Piano, I have no way to test this.
User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Miracle Piano in FCEUX

Post by Dwedit »

The data sent is pure MIDI messages, maybe not matching the actual wire protocol, but it does match MIDI messages.

When running a lesson, you should see the on-screen keyboard react when you play notes. It expects data to be read on MIDI channel 1, and outputs data on channels 1-8 I think.

There is both MIDI input and output to and from controller 1's port.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
GreyRogue
Posts: 51
Joined: Fri Dec 08, 2017 5:12 pm

Re: Miracle Piano in FCEUX

Post by GreyRogue »

The problem is probably this in MidiInitialize in Midi.cpp:

Code: Select all

	uint32 InDev = 9;
	uint32 OutDev = 19;
	const PmDeviceInfo *pmInfo = Pm_GetDeviceInfo(InDev);
	if (!miraclep->m_midiin.open_input(pmInfo->name))
		return 0;
	pmInfo = Pm_GetDeviceInfo(OutDev);
	if (!miraclep->m_midiout.open_output(pmInfo->name))
		return 0;
The big missing thing in the code is a way of inputting which MIDI devices to use.
If you look in Midi.cpp in the GetMidiDeviceList(), you'll see where it loops through all the available MIDI in and MIDI out devices and prints them, which you can see if you run fceux from a command line.

Alternatively, here's a different version looking for ones with "USB" in the name. Those matched my MIDI devices and successfully set InDev and OutDev (which I made global in this version) for use in MidiInitialize().

Code: Select all

/* FCE Ultra - NES/Famicom Emulator
*
* Copyright notice for this file:
*  Copyright (C) 2002 Xodnizel
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
*/

#include "common.h"
#include <Mmsystem.h> //(include Windows.h)
#pragma comment(lib, "winmm.lib")

#include "input.h"
#include "miracle.h"

static pm_module* InitPM = NULL;
nes_miracle_device* miraclep = NULL;

char MiraclePianoResponse[] = { 0xF0, 0x00, 0x00, 0x42, 0x01, 0x05, 0x01, 0x0A, 0xF7 }; // Response: Miracle Piano
char MiraclePianoQuery[] = { 0xF0, 0x00, 0x00, 0x42, 0x01, 0x04, 0xF7 }; // Query are you Miracle Piano
uint8 mididatain[0x200 * 10];// = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00 };
char TestNote[] = { 0x90, 0x30, 0x40 };
char mididataout[100];// = { 0xF0, 0x00, 0x00, 0x42, 0x01, 0x04, 0xF7 };
uint32 InDev = 9;
uint32 OutDev = 19;

void MidiUpdateState(void)
{
	miraclep->rcv_char();
}

void MidiClose(void)
{
	if (miraclep)
	{
		miraclep->m_midiin.close();
		miraclep->m_midiout.close();
		delete miraclep;
		miraclep = NULL;
	}
	if (InitPM)
	{
		InitPM->exit();
		InitPM = NULL;
	}
}

int GetMidiDeviceList()
{
	int num_devs = Pm_CountDevices();
	const PmDeviceInfo *pmInfo;

	printf("\n");

	if (num_devs == 0)
	{
		printf("No MIDI ports were found\n");
		return 0;
	}

	printf("MIDI input ports:\n");
	for (int i = 0; i < num_devs; i++)
	{
		pmInfo = Pm_GetDeviceInfo(i);

		if (pmInfo->input)
		{
			printf("%d %s %s\n", i, pmInfo->name, (i == Pm_GetDefaultInputDeviceID()) ? "(default)" : "");
			if (0 != strstr(pmInfo->name, "USB"))
				InDev = i;
		}
	}

	printf("\nMIDI output ports:\n");
	for (int i = 0; i < num_devs; i++)
	{
		pmInfo = Pm_GetDeviceInfo(i);

		if (pmInfo->output)
		{
			printf("%d %s %s\n", i, pmInfo->name, (i == Pm_GetDefaultOutputDeviceID()) ? "(default)" : "");
			if (0 != strstr(pmInfo->name, "USB"))
				OutDev = i;
		}

	}
	return 1;
}

int MidiInitialize(void)
{
	if (InitPM)
		return(1);

	InitPM = new pm_module();
	InitPM->init();
	miraclep = new nes_miracle_device();

	if (GetMidiDeviceList() == 0)
		return 0;

	const PmDeviceInfo *pmInfo = Pm_GetDeviceInfo(InDev);
	if (!miraclep->m_midiin.open_input(pmInfo->name))
		return 0;
	pmInfo = Pm_GetDeviceInfo(OutDev);
	if (!miraclep->m_midiout.open_output(pmInfo->name))
		return 0;

	for (int i = 0; i < sizeof(MiraclePianoQuery); i++)
		miraclep->m_midiout.write((MiraclePianoQuery[i]));
	Sleep(10);
	printf("pmRead=");
	while (miraclep->m_midiin.poll())
	{
		int BytesRead = miraclep->m_midiin.read(mididatain);
		for (int i = 0; i < BytesRead; i++)
			printf(" %02x", mididatain[i]);
		Sleep(1);
	}
	printf("\n");
	Sleep(1000);
	for (int i = 0; i < sizeof(TestNote); i++)
		miraclep->m_midiout.write((TestNote[i]));
	
	return 1;
}
You need to know what portmidi device number or device name you want to use for each to get it to work. The 9 and 19 were specific to my hardware setup. A GUI with a dropdown each for MIDI in and MIDI Out filled in by the results from GetMidiDeviceList() is the right way to do it.

I know it works with both an official Miracle Piano and a Yamaha MIDI keyboard, as I tested both (as well as software midi, as mentioned in my first post). The Yamaha sometimes used mismatched instruments, but was usable. It is possible to build a translation matrix from miracle piano instruments to standard MIDI instruments, but I didn't do this.
User avatar
DarkCecil
Posts: 5
Joined: Tue Dec 22, 2020 8:24 am

Re: Miracle Piano in FCEUX

Post by DarkCecil »

Thanks again for your reply, this was the missing piece of the puzzle for me! :) I modified those two values, did a rebuild, and immediately it stopped crashing.

I got my Yamaha keyboard with speakers built in working pretty well. The only issue is that midi output from the game plays AHEAD of the game/metronome, such as when in a song demo. Guess I'll have to figure out how to introduce some sort of delay to midi output.

Basically I too saw bits of TMR playing this game for 90 hours. I decided to challenge myself and see how I can compare to his time.

And yes, the songs sound very weird when played back through plain old GMidi. Maybe I'll create a soundfont or something.
User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Miracle Piano in FCEUX

Post by Dwedit »

Indeed, the instruments are NOT General Midi.
Nocash has the instrument list, as taken from the Miracle manual. Repeated here:

Code: Select all

  000 Grand Piano     032 Marimba         064 Synth Bells    096 Tube Bells'   
  001 Detuned Piano   033 Glockenspiel'   065 Vox 1          097 Frogs/Ducks   
  002 FM Piano        034 Kalimba'        066 Vox 2          098 Banjo'        
  003 Dyno            035 Tube Bells      067 Vox 3          099 Shakuhachi'   
  004 Harpsichord     036 Steel Drums     068 Mod Synth      100 Piano'        
  005 Clavinet        037 Log Drums'      069 Pluck Synth    101 Vibraphone'   
  006 Organ           038 Strings 1       070 Hard Synth     102 FM Piano'     
  007 Pipe Organ      039 Pizzicato       071 Syntar         103 Clock Belis'  
  008 Steel Guitar    040 Strings 2       072 Effects 1 *    104 Harpsichord'  
  009 12-StringGuitar 041 Violin 1'       073 Effects 2 *    105 Clavinet'     
  010 Guitar          042 Trumpet'        074 Percussion 1 * 106 Organ'        
  011 Banjo           043 Trumpets        075 Percussion 2 * 107 Pipe Organ'   
  012 Mandolin        044 Horn'           076 Percussion 3 * 108 Metal Guitar' 
  013 Koto'           045 Horns           077 Sine Organ'    109 Stick'        
  014 Jazz Guitar'    046 Trombone'       078 Organ #        110 Guitar'       
  015 Clean Guitar'   047 Trombones       079 Pipe Organ #   111 Xylophone'    
  016 Chorus Guitar   048 CupMuteTrumpet' 080 Harpsichord #  112 Marimba'      
  017 Fuzz Guitar     049 Sfz Brass 1     081 Synth Pad 1    113 Syn Trombone' 
  018 Stop Guitar     050 Sfz Brass 2     082 Synth Pad 2    114 Syn Trumpet'  
  019 Harp'           051 Saw Synth       083 Synth Pad 3    115 Sfz Brass 1'  
  020 Detuned Harp    052 Tuba'           084 Synth Pad 4    116 Sfz Brass 2'  
  021 Upright Bass'   053 Harmonica       085 Synth Pad 5    117 Saw Synth'    
  022 Slap Bass'      054 Flute'          086 Synth Pad 6    118 Church Bells' 
  023 Electric Bass'  055 Pan Flute'      087 Synth Pad 7    119 Marcato'      
  024 Moog            056 Calliope        088 Synth Pad 8    120 Marcato       
  025 Techno Bass     057 Shakuhachi      089 Synth Pad 9    121 Violin 2'     
  026 Digital Waves   058 Clarinet'       090 Synth Pad 10   122 Strings 3     
  027 Fretless Bass'  059 Oboe'           091 Synth Pad 11   123 Synth Bells'  
  028 Stick Bass      060 Bassoon'        092 Synth Pad 12   124 Techno Bass'  
  029 Vibraphone      061 Sax'            093 Synth Pad 13   125 Mod Synth'    
  030 MotorVibraphone 062 Church Bells    094 Synth Pad 14   126 Pluck Synth'  
  031 Xylophone       063 Big Bells       095 Synth Pad 15   127 Hard Synth'   
  
' = max 16 notes (otherwise 8)
* = see effects/percussion list
# = ignores velocity

Effects/Percussion:

  Note      Effects 1   Effects 2    Percussion 1   Percussion 2   Percussion 3
  30-35     Jet         Yes (ding)   -              -              Ratchet
  36-4l     Gunshot     No (buzz)    Kick Drum      Rim Shot       Snap 1
  42-47     RoboDeath   Applause     Snare          Exotic         Snap 2
  48-53     Whoosh      Dogbark      Toms           Congas         Dripdrum 1
  54-59     Punch       Door creak   Cymbal         Timbale        Dripdrum 2
  60-65     Slap        Door slam    Closed Hat     Cowbell        Wet clink
  66-71     Duck        Boom         Open Hat       Bongos         Talk Drum
  72-77     Ow! 1       Car skid     Ride           Whistle        Agogo
  78-83     Ow! 2       Goose        Shaker         Clave          Explosion
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
User avatar
DarkCecil
Posts: 5
Joined: Tue Dec 22, 2020 8:24 am

Re: Miracle Piano in FCEUX

Post by DarkCecil »

I have created a SoundFont for use with the Miracle Piano.

https://github.com/DarkCecil82/Miracle-Piano-SoundFont

Feel free to try it out, make changes, suggestions, etc.

At first, I used it with CoolSoft VirtualMidiSynth, but that had too much latency for my tastes...
So then I moved over to the ASIO side of things and used my DAW (Cantabile Performer) with BassMidi VSTi. I routed FCEUX's MIDI output to Midi-Yoke, and fed that over to BassMidi as well. I then took that output as a whole and fed it to my cheap Behringer USB interface + headphones. This allowed me to get near-zero latency with my USB keyboard.

Now the problematic part -- The game's metronome. It was lagging behind everything else. I believe it is isolated to FCEUX. Let's say I pick a song and have the game demonstrate it -- The metronome is heard late. Or let's say I try to play along to a song -- I have to play at least 50ms AHEAD of the metronome to get the game to give me a pass. No matter how I route the audio out from FCEUX, the result is the same. It's so bad for some lessons that I have to mute the metronome and follow the arrow on-screen while playing, so video seems to be fine. That works well with moderate-fast tempos. Not so much with really slow ones.

I also experienced this same problem when using my Yamaha USB keyboard with built-in speakers/audio, similar to an actual Miracle. Except here, I can't use a SoundFont to fix the instruments.

So I think this is the brick wall for now. I made it through the first 10 sections in the game in about 3 hours, but it wasn't easy having to compensate for the lag. The Lua script for 07.05 worked out great, though -- Thanks GreyRogue! I plan to get my hands on an actual Miracle pretty soon and try it out on my MiSTer and actual NES, and have something to compare to.
User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Miracle Piano in FCEUX

Post by Dwedit »

It's not quite a brick wall yet, there are still ways to compensate for audio playing late. This is where RunAhead comes in. If the audio is 50ms late, you can Run Ahead 3 frames to make up for that.

I know that FCEUX doesn't have RunAhead implemented in there, but it's like 10 lines of code to add in the feature (plus fixing audio on loadstate, maybe I should send in a patch to add RunAhead into FCEUX)

What is the current main repository for taking Pull Requests? Looks like it's the TASVideos repository.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
GreyRogue
Posts: 51
Joined: Fri Dec 08, 2017 5:12 pm

Re: Miracle Piano in FCEUX

Post by GreyRogue »

DarkCecil wrote: Fri Dec 25, 2020 10:02 am So I think this is the brick wall for now. I made it through the first 10 sections in the game in about 3 hours, but it wasn't easy having to compensate for the lag. The Lua script for 07.05 worked out great, though -- Thanks GreyRogue! I plan to get my hands on an actual Miracle pretty soon and try it out on my MiSTer and actual NES, and have something to compare to.
I did add support for USB Midi devices as a MiraclePiano input/output on MiSTer a while back, which I tested with both a Yamaha keyboard and a Miracle Piano (with a USB adapter for both), so you can probably use it now if you want (don't have to wait for the real Miracle Piano). I didn't test it much, but I know it worked. If you want to use SNAC with a Miracle Piano cable to connect instead of USB Midi to the MiSTer, you do need a real Miracle Piano, though. The Yamaha will have the same instrument mismatch issue as FCEUX, unless you use your custom setup for Midi Output with your soundfont (ie, not using the Yamaha as sound generation; Midi Input only).
Post Reply