HDMI SNES?

Discussion of hardware and software development for Super NES and Super Famicom. See the SNESdev wiki for more information.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
User avatar
jwdonal
Posts: 719
Joined: Sat Jun 27, 2009 11:05 pm
Location: New Mexico, USA
Contact:

Re: HDMI SNES?

Post by jwdonal »

lidnariq wrote:Injecting the correct 768×32000 Hz clock is the only correct answer.
It's not the only correct answer. There are multiple perfectly valid ways to accomplish this task as I mentioned above - just as there are in essentially every engineering task. Try telling a team of engineers in industry, "This is the one and only correct way to accomplish this task" and you'll get some good laughs.

But assuming by "injecting" you mean tapping into the console's 24.576MHz osc and hooking it up to the FPGA you can certainly do that like I said earlier (with caveats), but you probably don't want to drive the actual HDMI/SPDIF controller/output using that source clock. In fact, if you're using an external HDMI output controller chip you're guaranteed it's going to require some very specific clock input that is very likely not 24.576MHz. Lol. So no matter what you do you're going to have to cross clock domains at some point. Therefore, the best design practice would be to capture the samples into a bi-synchronous FIFO/RAM within the FPGA at the source sampling rate and then read those samples out of the FIFO using a completely separate clock generated internal to the FPGA using one of the FPGA's internal PLL cores so you can get a nice clean, accurate, and stable clock source to drive it out to its next destination.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: HDMI SNES?

Post by lidnariq »

Resampling is such a tremendously more expensive operation than just giving it the correct clock in the first place that I honestly can't fathom how you're seriously arguing this.

You'd need to reclock everything in the SNES (much as Kevtris had to in the NES) to produce an HDMI-compliant signal. So you set the entire thing up to generate the data at the right rates in the first place. Otherwise you drop or interpolate samples or frames, and that's unambiguously suboptimal. Nevermind that downsampling can be a lossy operation, and at 32kHz we can hear the entire spectral content so losing anything would be unfortunate.
jwdonal wrote:But assuming by "injecting" you mean tapping into the console's 24.576MHz osc and hooking it up to the FPGA you can certainly do that like I said earlier (with caveats), but you probably don't want to drive the actual HDMI/SPDIF controller/output using that source clock.
Why on earth would you assume that "inject" means "use"? Why would I talk about "injecting exactly 32000 × 768 Hz" if I meant "use the output of the ceramic resonator"?
93143
Posts: 1717
Joined: Fri Jul 04, 2014 9:31 pm

Re: HDMI SNES?

Post by 93143 »

How would you handle Super Game Boy and MSU1?
User avatar
jwdonal
Posts: 719
Joined: Sat Jun 27, 2009 11:05 pm
Location: New Mexico, USA
Contact:

Re: HDMI SNES?

Post by jwdonal »

lidnariq wrote:Resampling is such a tremendously more expensive operation than just giving it the correct clock in the first place that I honestly can't fathom how you're seriously arguing this.
Um..say what? What tremendous amount of resources do you believe re-sampling requires? It's like 3 lines of verilog. Lol.
lidnariq wrote:You'd need to reclock everything in the SNES (much as Kevtris had to in the NES) to produce an HDMI-compliant signal. So you set the entire thing up to generate the data at the right rates in the first place. Otherwise you drop or interpolate samples or frames, and that's unambiguously suboptimal. Nevermind that downsampling can be a lossy operation, and at 32kHz we can hear the entire spectral content so losing anything would be unfortunate.
Yeah...so none of that makes any sense. In any case, if you're using an HDMI controller like the AD9889B for example, you will have to resynchronize all of the data to a 165MHz clock. You don't just pick your own clock rates for Industry standard audio/video interface devices. :P
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: HDMI SNES?

Post by lidnariq »

Really, it's just three lines of verilog for an arbitrary and dynamic sample rate converter that you can't know before you boot up? Because, you know, ceramic resonators and their sucking. You should expect drift as it gets warm, and you should significant age-based drift too. Just in case it's not obvious, I mean one that doesn't generate audible artifacts, none of this lousy "sample and hold interpolation" or "linear interpolation" BS.

Anyway, I still don't see why you're seriously arguing that resampling's a right way to do it. (Does it work? Of course it "works". So what? ZSNES "works" too.) Why on earth would you fail to put everything in the SNES on the same clock domain in the first place?
User avatar
jwdonal
Posts: 719
Joined: Sat Jun 27, 2009 11:05 pm
Location: New Mexico, USA
Contact:

Re: HDMI SNES?

Post by jwdonal »

lidnariq wrote:Really, it's just three lines of verilog for an arbitrary and dynamic sample rate converter that you can't know before you boot up?
Yes, that's exactly what I'm saying.
lidnariq wrote:Because, you know, ceramic resonators and their sucking. You should expect drift as it gets warm, and you should significant age-based drift too. Just in case it's not obvious, I mean one that doesn't generate audible artifacts, none of this lousy "sample and hold interpolation" or "linear interpolation" BS.
You're making this incredibly more complex than it actually needs to be. What I was trying to explain before is that you wouldn't need to worry about any of that if you just tapped into the console's audio clock. Then you're re-sampler would always be using that console's exact source frequency. So it'll always be perfect. Or if you want to use your own clock you just need to make sure you re-sample at at least 64kHz (Nyquist) and you'd be fine - you could even go a tad faster if you're really that worried about slightly faster oscillator frequencies from console to console. But personally I wouldn't even bother.

UPDATE: So just as a test I tied the analog audio out of my FPGA SPC player running on my DE2-115 and tied it into the analog audio input of my ML507, re-sampled the output with an internally generated 64kHz clock (and sorry, I lied, it was 4 lines of verilog, so sue me) and then drove those samples back out of the ML507's audio codec and hooked it up to some speakers. It sounded perfectly fine to me. Although I really didn't need to do any of that to know it would work - Nyquist/Shannon proved it would work back in the 1950s or something. Heh. ;) Hell, I probably could have done it in 0 lines of verilog if I just let the codec do all the work for me. Lolz. So I'm sorry, but for all your fancy technical jargon it's really just as simple as I said it was. I would also keep in mind that the original SNES audio source wasn't that high fidelity to begin with due to all of the compression/decompression/interpolation it had to go through before it even got to your ears. Bleh.

But if you really want to make this insane Rube Goldberg audio thinga-ma-jigger whatcha-ma-call-it then more power to ya! 8-)
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: HDMI SNES?

Post by lidnariq »

You should be careful about "I can't hear anything wrong" tests with audio stuff.

Since you say it's simple enough to be just four lines of verilog, it seems a safe bet that you're describing sample-and-hold interpolation. So here's what happens when you take a pure 1kHz sine at 32ksps; parse it as 32032sps 31968sps (edit: braino), and upsample that to 96000sps using this sample-and-hold interpolation:
sampleandhold.ogg
(50.99 KiB) Downloaded 181 times
The spectrogram (there's only about 40dB of rejection here, ignoring the ultrasonic portion):
sampleandhold.png
The residual:
residual.ogg
(57.16 KiB) Downloaded 189 times
Just because you can't hear it doesn't mean the artifacts aren't there, and especially doesn't mean that someone else won't hear it. And that's why I said that the correct solution is using the correct clock in the first place, rather than something that produces resampling artifacts.

Here's the matlab/octave code I used to generate these:

Code: Select all

inidex=[0:31999];
ssin=sin(inidex*pi/16);
outidex=[0:.333:31999];
ssout=interp1(inidex,ssin,outidex,"nearest");
loglog(abs(fft(ssout))(1:48000));
ssquant=round(32767*ssout);
save("sampleandhold.txt","ssquant");
ffout=fft(ssout);
ffout(1001) = ffout(95095) = 0;
iiout=ifft(ffout);
iiquant=round(real(32767*iiout));
save("residual.txt","iiquant");
[/size]
Last edited by lidnariq on Tue Oct 27, 2015 12:08 pm, edited 1 time in total.
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: HDMI SNES?

Post by tepples »

Let's assume an adapter uses the Hi-Def NES approach, with the PPU feeding the upscaler through a circular buffer a couple dozen lines tall. Let's further assume that it doesn't provide for Super Game Boy audio. The adapter would feed a slightly slow master clock to the S-CPU and S-PPUs such that the frame rate slows from 60.10 to 60.0 or 59.94 Hz or whatever the TV expects. It'd also feed a precisely 24.576 MHz clock into the audio module so that the 32000 Hz output has a small integer ratio to what the TV expects. (A cubic spline interpolation circuit is probably easier to build with such a small integer ratio.)

At an average 1364*261+1362 = 357366 master clocks per frame, the S-CPU and S-PPUs would be fed a 21.44196 MHz clock to produce exactly 60.00 Hz, which is one part in 608 slow. If the APU is fed 24.576 MHz exactly, then from the game's perspective, the CPU:APU clock ratio would imply 32053 Hz, which is within observed limits.

Have there been experiments with under- or overclocking the audio module by replacing the 24.576 MHz oscillator with a variable frequency one to determine how games (mis)behave? Or perhaps putting the master clock through an 8/7 multiplier to produce 24.545 MHz?
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: HDMI SNES?

Post by tepples »

Brainstorming a bit more about this mythical HDMA (high definition multimedia adapter):

The ADC to digitize Super Game Boy audio for mixing into the HDMI output wouldn't even need to be real expensive, as the Super Game Boy's output is about 6 bits per channel so you can get away without a lot of SNR. MSU1, on the other hand, might need something more expensive in order not to introduce noise.
User avatar
mikejmoffitt
Posts: 1353
Joined: Sun May 27, 2012 8:43 pm

Re: HDMI SNES?

Post by mikejmoffitt »

tepples wrote:Let's assume an adapter uses the Hi-Def NES approach, with the PPU feeding the upscaler through a circular buffer a couple dozen lines tall. Let's further assume that it doesn't provide for Super Game Boy audio. The adapter would feed a slightly slow master clock to the S-CPU and S-PPUs such that the frame rate slows from 60.10 to 60.0 or 59.94 Hz or whatever the TV expects. It'd also feed a precisely 24.576 MHz clock into the audio module so that the 32000 Hz output has a small integer ratio to what the TV expects. (A cubic spline interpolation circuit is probably easier to build with such a small integer ratio.)
Forgetting audio for a moment (since unlike NTSC video, analogue audio hasn't aged or become as difficult to utilize effectively) aren't we still stuck on getting digital pixel information from the PPU? That seems like the first hurdle that must be crossed.
User avatar
defparam
Posts: 35
Joined: Sat Oct 04, 2014 3:48 pm

Re: HDMI SNES?

Post by defparam »

So when I was hacking around with the expansion port (see snes-tap) I was able to passively snoop all reads and writes on route to the CPU/APU from my FPGA. With all this data snooped I had the FPGA forward it to my laptop through a FTDI USB 2.0 device. On my laptop I had a re-compiled version of Snes9x with the ability to grab the console data and emulate the video to show video representation of the console in emulator. I didn't spend too much time on it but the audio wouldn't be too much more work to get it going. In any case my point is one solution to the problem could be to take address bus B read/writes and use a 3rd party microcontoller (like a raspberry pi since it has HDMI) to perform snes audio/video emulation for the purpose of HDMI.
lidnariq
Posts: 11432
Joined: Sun Apr 13, 2008 11:12 am

Re: HDMI SNES?

Post by lidnariq »

Given what you (mikejmoffitt) pointed out near the beginning of the thread, it seems likely that we could "just" monitor the inter-PPU bus ... but it seems likely that we'd still have to emulate PPU2.

Do we have an idea of what the division of labor is between PPU1 and PPU2 is?
User avatar
mikejmoffitt
Posts: 1353
Joined: Sun May 27, 2012 8:43 pm

Re: HDMI SNES?

Post by mikejmoffitt »

defparam wrote:So when I was hacking around with the expansion port (see snes-tap) I was able to passively snoop all reads and writes on route to the CPU/APU from my FPGA. With all this data snooped I had the FPGA forward it to my laptop through a FTDI USB 2.0 device. On my laptop I had a re-compiled version of Snes9x with the ability to grab the console data and emulate the video to show video representation of the console in emulator. I didn't spend too much time on it but the audio wouldn't be too much more work to get it going. In any case my point is one solution to the problem could be to take address bus B read/writes and use a 3rd party microcontoller (like a raspberry pi since it has HDMI) to perform snes audio/video emulation for the purpose of HDMI.
That's a possibility, but as that involves re-implementing the PPU and APU, it would be best to explore how much we can pull directly. There is also a discussion of latency that must be had with a higher level system like that.
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Re: HDMI SNES?

Post by Near »

The expansion port approach would require full PPU1+PPU2 emulation. But would be mod-free. It should produce a perfect image. The only issue would be added latency (it depends on how tightly you control the HDMI output. If you're using an RPi, it'd be as bad as a PC emulator's latency.)

If you were going to physically mod the consoles to remove chips, then I have no idea what you could do, but good luck. There's little documentation about all the exact interactions between the two PPUs, and later SNES revisions merge the PPUs into the same IC as the CPU.

Personally, my money's on an FPGA implementation of the PPUs through the expansion port, with a PC-emulator implementation as the proof-of-concept first. A mod-free adapter for HDMI video would sell like hotcakes. People are already doubling the price of SNES consoles with 1CHIPs for their ~10% better analog RGB quality. This adapter would obsolete nearly all of that market, and likely be even more valuable than the HDMI NES mods, but cost substantially less to produce.

The first person to market could probably easily pull in $200 a board, SNES console not included. The video quality would be vastly superior to even the XRGB, but you'd still have to charge less since it's an SNES-only solution. The NES HDMI mods have already shown that people don't care if the video processor alone is emulated, if it gets them crisp digital pixels.

Also, when it comes to audio ... the expansion port only carries analog mono. You would have to:
a) emulate the SMP+DSP as well to produce bit-perfect stereo audio (S/PDIF or the like)
b) grab the mono output only and output to both speakers
c) connect the expansion port board to the MultiAV to steal its audio signal
(c could also be a more stable source of Vblank timing than trying to snoop NMI through vector reads)
tepples
Posts: 22708
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: HDMI SNES?

Post by tepples »

Once you're emulating the S-PPU1, S-PPU2, S-SMP, and S-DSP, you might as well emulate the CPU and DMA too and make a Super Famiclone.
Post Reply