Emulators with video capture and good sound emulation

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
psycopathicteen
Posts: 2904
Joined: Wed May 19, 2010 6:12 pm

Emulators with video capture and good sound emulation

Post by psycopathicteen » Mon Feb 13, 2017 4:14 pm

I usually use BSNES for play testing, but SNES9X for YouTube videos, but it sounds like SNES9x doesn't do good pitch modulation. Am I just using an outdated emulator, or is SNES9x still like that. Or is pitch modulation supposed to sound like that, and BSNES sounds wrong?

Kismet
Posts: 57
Joined: Wed Nov 30, 2016 9:59 pm

Re: Emulators with video capture and good sound emulation

Post by Kismet » Mon Feb 13, 2017 6:30 pm

psycopathicteen wrote:I usually use BSNES for play testing, but SNES9X for YouTube videos, but it sounds like SNES9x doesn't do good pitch modulation. Am I just using an outdated emulator, or is SNES9x still like that. Or is pitch modulation supposed to sound like that, and BSNES sounds wrong?
None of them. The problem isn't the emulator, it's the capture mechanics that people use. Snes9x is the only emulator that actually can capture bitstream video and audio in the emulator without skipping frames, but you have to do it by first recording the SMV file, and then playing it back with the avi recording.

You need to capture the video with zmbv codec, integer rescale it to at least 720p60 and then upload the avi directly to youtube. Otherwise youtube will convert 240p60 into 360p30 and cut the audio fidelity by about half.

This is a lot of work to make sure the resulting output isn't garbage. If people use any of the standard video capture tools (eg OBS, Xplit) the video and audio will be lossy compressed by the capture software as it will record at the desktop volume, thus requiring normalizing the audio.

This is why youtube videos of emulated games tend to be terrible, because the video uploaded was compressed with h.264 and aac audio at lossy rates, and then youtube recompresses it a second time.
There is no way to get youtube to play lossless video/audio. However if you upload a lossless source (1440x1080 ZMBV 32bpp, keyframe roughly every 600 frames) you actually will run into a file size limit where youtube will flake out when it crosses about 2GB of data. So that's why I end up cutting videos into several pieces.

So from a preservation point of view, you can have the best emulator in the world, but if you capture it poorly, garbage in, garbage out. It is technically possible to capture h.264(4:4:4) in a lossless mode, but not many video tools can understand this as most h.264 video is 4:2:0. h.264 is what most USB-based capture devices use, and are likewise much more lossy than what you can do with an emulator.

I would actually love to have higan support a mechanic like the SMV file so that it could render an AVI, or invoke ffmpeg so that the resulting file could be uploaded to youtube in one step, but I don't think very many people have this use case in mind since video editing tools tend to balk at frame-based video and all want to work with time-based video (mpeg-2 or mpeg4 containers.)

TL;DNR version:
You either have to use an emulator that can record losslessly or you can use a high-accuracy emulator, but end up having to use lossy capture tools, so neither case is going to have the right audio.
I come from the net. Through systems, peoples and cities to this place.

tepples
Posts: 21752
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Emulators with video capture and good sound emulation

Post by tepples » Mon Feb 13, 2017 6:52 pm

Pad to 280x240, then scale nearest-neighbor to 560x480. If you pixel double before encoding, the 4:2:0 crap won't hurt. This leaves only YouTube's 30 fps limit on video smaller than HD, but that can be worked around by either frame-blending (so that flicker is handled in a least-astonishing manner) or exhibiting your video elsewhere.

I don't know what to use with Super NES emulation, but this is what I came up with for NES using FCEUX (SDL) and FFmpeg on Linux:

Code: Select all

fceux \
  --xscale 1 --yscale 1 --special 0 \
  --pal 0 \
  --sound 1 --soundq 1 --soundrate 48000 --mute 1 \
  --nospritelim 0 \
  --no-config 1 \
  --videolog "ffmpeg -y \
  -f rawvideo -r 60.0988 -pix_fmt bgr24 -s 256x224 -i v.bgr \
  -f s16le -ar 48000 -i s.log \
  -vf \"tblend=all_mode=average,pad=280:240:12:8,setsar=8:7,scale=w=560:h=480:flags=neighbor\" \
  -t 45 -r 30 out1.mp4" \
  --playmov some_tas.fm2 some_rom.nes
Or when making a lossless intermediate, replace out1.mp4 with something like -vcodec huffyuv -acodec copy out1.avi

Any chance of a bsnes-rerecording?

lidnariq
Posts: 8785
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Emulators with video capture and good sound emulation

Post by lidnariq » Mon Feb 13, 2017 6:55 pm

tepples wrote:Any chance of a bsnes-rerecording?
I think that's called LSNES, if I understood correctly.

psycopathicteen
Posts: 2904
Joined: Wed May 19, 2010 6:12 pm

Re: Emulators with video capture and good sound emulation

Post by psycopathicteen » Mon Feb 13, 2017 9:09 pm

You guys missed the point entirely. SNES9x sounds bad even when I'm not recording a video. I was asking if there was either a way to fix SNES9x's sound quality, or a video capture plugin for other emulators.

tepples
Posts: 21752
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Emulators with video capture and good sound emulation

Post by tepples » Mon Feb 13, 2017 9:20 pm

As far as I can tell, the comments addressed the feasibility of "video capture plugin for other emulators". In particular, I mentioned "rerecording" because an emulator with thorough support for key logs would be amenable to modification to support recording of a key log, such as Snes9x's SMV file, and then video capture from that key log.

Kismet
Posts: 57
Joined: Wed Nov 30, 2016 9:59 pm

Re: Emulators with video capture and good sound emulation

Post by Kismet » Mon Feb 13, 2017 10:19 pm

psycopathicteen wrote:You guys missed the point entirely. SNES9x sounds bad even when I'm not recording a video. I was asking if there was either a way to fix SNES9x's sound quality, or a video capture plugin for other emulators.
Well, the ideal solution is retrofit Higan
1) retrofit higan with SMV support
2A) retrofit higan with either the ZMBV codec ala DOSBOX, or
2B) retrofit higan with being able to pipe a raw stream (-c:v rawvideo -c:a pcm_16le) to FFMPEG

FFMPEG and Youtube can read the ZMBV 16bpp or 32bpp without any trouble. Only the 8bpp indexed color mode has any minor issues, which you wouldn't use for SNES emulation anyway.

What I used with DOSBOX if I'm trying to force Youtube to not suck
ffmpeg -i "%1" -g 30 -coder 1 -bf 2 -flags +cgop -vcodec libx264 -x264opts ref=5:opencl:crf=22.0 -pix_fmt yuv420p -profile:v high -level 52 -r 60 -movflags faststart -codec:a aac -b:a 512k -f mp4 %1.mp4
This doesn't include scaling or aspect ratio correction, as typically I would cut/edit/crop/scale the video in virtualdub first. If I bypass that, it's instead
ffmpeg -i "%1" -sws_flags neighbor+full_chroma_inp -vf scale=-1:2160 -g 30 -coder 1 -bf 2 -flags +cgop -vcodec libx264 -x264opts ref=5:opencl:crf=22.0 -pix_fmt yuv420p -profile:v high -level 52 -r 60 -movflags faststart -codec:a aac -b:a 512k -f mp4 %1.mp4
Needless to say these settings are extremely overkill and are meant really for video editing rather than straight to youtube, but if you upload it straight to youtube, it will work. This doesn't include aspect ratio correction or padding (see tepples example)

Basically youtube is a crapshoot, because they change things sometimes, and that results in files sometimes work when they are uploaded and stop working later. For example, I have a few videos I used the ZMBV codec for on youtube with no keyframes, and originally Youtube transcoded to all smaller formats automatically (eg 720p to 480p) and this worked fine, but then youtube seems to have switched to transcode-on-the-fly and without keyframes, you can't seek these videos. Likewise 4:4:4 h264 video will upload, and so will non-compliant frame-rates and levels will upload, but they will play at the wrong speed or will have no picture after the first 2 seconds.

The worst thing I discovered with youtube is that it drops frames during playback, so if you seek in youtube to say 1:00 it will not be 1:00 in the original video.

I know this doesn't address "why does the sound not sound accurately" question but what I'm trying to point out here is that there is not currently a way to capture accurate audio in an emulator, and even if you could, the average person doesn't know how to maintain that audio over their editing process, and then youtube will ruin it if you don't upload the footage at 720p60.
I come from the net. Through systems, peoples and cities to this place.

creaothceann
Posts: 210
Joined: Mon Jan 23, 2006 7:47 am
Location: Germany
Contact:

Re: Emulators with video capture and good sound emulation

Post by creaothceann » Tue Feb 14, 2017 1:37 am

Just use BizHawk, it uses the bsnes core.

http://tasvideos.org/Bizhawk/SNES.html
http://tasvideos.org/forum/viewforum.php?f=64
http://tasvideos.org/forum/viewforum.php?f=52

I usually resize the video with Avisynth to >= 720p and encode the video/audio with x264 (or x264vfw in VirtualDub) and the audio with 192kbps MP3.
My current setup:
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10

Kismet
Posts: 57
Joined: Wed Nov 30, 2016 9:59 pm

Re: Emulators with video capture and good sound emulation

Post by Kismet » Tue Feb 14, 2017 4:33 am

creaothceann wrote:Just use BizHawk, it uses the bsnes core.

http://tasvideos.org/Bizhawk/SNES.html
http://tasvideos.org/forum/viewforum.php?f=64
http://tasvideos.org/forum/viewforum.php?f=52

I usually resize the video with Avisynth to >= 720p and encode the video/audio with x264 (or x264vfw in VirtualDub) and the audio with 192kbps MP3.
I'd suggest staying away from multi-emulator suites that don't write their own cores, because they tend to never update the libraries once they get them working because they're afraid of breaking them afterwords. In the case of Bizhawk it uses BSNES v87 from 2012.

Lsnes uses BSNES v85 from 2013.

(Those are the git import dates that I could find)

x264vfw/mp3 is probably the worst possible option (as it just keyframes h264 frames from a very old version of x264 from 2009.) ZMBV (from dosbox) produces the smallest lossless files for emulators. Lagarith or huffyuv produce much larger files because they are meant to be lossless intermediate files that you record to because raw uncompressed captures are too large for mechanical drives at HD rates.

Another codec that can be used as an intermediate is the camstudio codec but there are bugs in some versions of it so you have to use the same version of the codec to record and play, and not switch between the stand alone codec and the ffdshow version.

That's why I use the ZMBV codec for short things that can be broken up into chapters like RPG's. If you want to do one long video, your only option is h.264/aac or h.264/pcm but the latter, the audio is 8.8MB/minute alone.
I come from the net. Through systems, peoples and cities to this place.

calima
Posts: 1017
Joined: Tue Oct 06, 2015 10:16 am

Re: Emulators with video capture and good sound emulation

Post by calima » Tue Feb 14, 2017 4:57 am

Realtime encoding tends to put too much strain on the cpu, potentially causing jitter. What I did for dgen, when I needed Genesis recording, was add dumping support as individually LZO-compressed frames. This resulted in minimal dumping overhead, and quite acceptable space usage as well. For audio, I just told alsa to dump what it received to a wav file.

Kismet
Posts: 57
Joined: Wed Nov 30, 2016 9:59 pm

Re: Emulators with video capture and good sound emulation

Post by Kismet » Tue Feb 14, 2017 5:11 am

calima wrote:Realtime encoding tends to put too much strain on the cpu, potentially causing jitter. What I did for dgen, when I needed Genesis recording, was add dumping support as individually LZO-compressed frames. This resulted in minimal dumping overhead, and quite acceptable space usage as well. For audio, I just told alsa to dump what it received to a wav file.
The Camshow codec uses lzo and delta frames (the bugs in some versions are the delta frames playback), ZMBV uses straight lz (via zlib.)

And yes, what you really want with software emulation is to playback the state of the emulator, not in real time, so it renders everything, at the correct framerate, and this is something that higan would be incredibly useful for.
I come from the net. Through systems, peoples and cities to this place.

creaothceann
Posts: 210
Joined: Mon Jan 23, 2006 7:47 am
Location: Germany
Contact:

Re: Emulators with video capture and good sound emulation

Post by creaothceann » Tue Feb 14, 2017 7:02 am

Kismet wrote:BSNES v87 from 2012
And isn't that good enough?
Kismet wrote:x264vfw/mp3 is probably the worst possible option (as it just keyframes h264 frames from a very old version of x264 from 2009.) ZMBV (from dosbox) produces the smallest lossless files for emulators.
I use ZMBV for recording (iirc BizHawk doesn't compress the audio). The x264vfw/mp3 combo is just for YT/Vimeo.
My current setup:
Super Famicom ("2/1/3" SNS-CPU-GPM-02) → SCART → OSSC → StarTech USB3HDCAP → AmaRecTV 3.10

User avatar
TmEE
Posts: 712
Joined: Wed Feb 13, 2008 9:10 am
Location: Estonia, Rapla city (50 and 60Hz compatible :P)
Contact:

Re: Emulators with video capture and good sound emulation

Post by TmEE » Tue Feb 14, 2017 8:05 am

Kega Fusion uses a custom codec that essentially just stores whatever data is written to the video chips, resulting in really tiny lossless files and no meaningful extra CPU power during recording. When done you convert the file over with your favorite video processing software. That's probably a bit more work on software side (new video codec creation is probably not straightforward) than creating and then recording keylogs into a video.

Kismet
Posts: 57
Joined: Wed Nov 30, 2016 9:59 pm

Re: Emulators with video capture and good sound emulation

Post by Kismet » Tue Feb 14, 2017 7:26 pm

TmEE wrote:Kega Fusion uses a custom codec that essentially just stores whatever data is written to the video chips, resulting in really tiny lossless files and no meaningful extra CPU power during recording. When done you convert the file over with your favorite video processing software. That's probably a bit more work on software side (new video codec creation is probably not straightforward) than creating and then recording keylogs into a video.
That's basically what ZMBV does in indexed mode on dosbox, the changes to video memory are delta frames, and changes to palette are basically the only thing different in many frames.

A SNES-specific codec in a mpeg-2 or mpeg-4 container could actually store the proper framerate if it's time-based. However because the SNES output is > 256 colors, you wouldn't be able to just store the changes to the video memory because how the SNES works is closer to having 5 layers with color math (transparencies)

Basically the ideal mechanic is to use something like snes9x or performance-based bsnes forks to record the input, and then play back the input on higan to a lossless+scaled+aspect ratio correct output.
creaothceann wrote:
Kismet wrote:BSNES v87 from 2012
And isn't that good enough?
That's good enough for TAS, that's not good enough for quality. If you're trying to preserve the output, especially for mastering for another use, you need to use the most accurate builds.
creaothceann wrote:
Kismet wrote:x264vfw/mp3 is probably the worst possible option (as it just keyframes h264 frames from a very old version of x264 from 2009.) ZMBV (from dosbox) produces the smallest lossless files for emulators.
I use ZMBV for recording (iirc BizHawk doesn't compress the audio). The x264vfw/mp3 combo is just for YT/Vimeo.


I would never use mp3 for youtube because youtube will always transcode that to aac.

The examples I used above

Code: Select all

 -g 30 -coder 1 -bf 2 -flags +cgop -vcodec libx264 -x264opts ref=5:opencl:crf=22.0 -pix_fmt yuv420p -profile:v high -level 52 -r 60 -movflags faststart -codec:a aac -b:a 512k -f mp4 
That's based on settings that youtube recommends and thensome, a more conservative setting is
https://www.virag.si/2015/06/encoding-v ... th-ffmpeg/

Code: Select all

-codec:v libx264 -crf 21 -bf 2 -flags +cgop -pix_fmt yuv420p -codec:a aac -strict -2 -b:a 384k -r:a 48000 -movflags faststart
The difference between the above and the below is CRF goes from 22 to 21, and audio goes from 384 to 512, and Google's recommended settings are on https://support.google.com/youtube/answer/1722171?hl=en
I come from the net. Through systems, peoples and cities to this place.

lidnariq
Posts: 8785
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Emulators with video capture and good sound emulation

Post by lidnariq » Tue Feb 14, 2017 9:07 pm

Kismet wrote:youtube will always transcode that to aac.
And vorbis. And opus. And you don't know which encoding or bitrate the viewer will get.

Chiptunes are a particularly bad match to any lossy codec at all; with some I can pass an ABX test even at bitrates well above the nominal "normal" audio "transparency" bitrate. The only conclusion I can come to, since you can't specify the codec, is to just not worry about it much.

Post Reply