video roms

Discussion of hardware and software development for Super NES and Super Famicom.

Moderator: Moderators

Forum rules
  • For making cartridges of your Super NES games, see Reproduction.
Post Reply
smkd
Posts: 101
Joined: Sun Apr 22, 2007 6:07 am

video roms

Post by smkd » Tue Nov 24, 2009 9:48 pm

I was recently messing around with trying to get the SNES to display smooth video, I had to make huge comprimises but I got a clip of me playing F-Zero at full speed running here.

Process was PJ64->fraps->vdub->irfanview->a program I made to convert / interleave pixels->incbin in xkas.

Random stats are 7.12MB for the 64x56 4bit video, 830KB for the 22khz audio (thanks snesbrr) and about 40kbish or something for the code. It takes about 55% of a frame or so to unpack video and prepare the audio, I couldn't get stuff to fit in a frame until I realised I could shave off a huge amount of code just by interleaving and duplicating the 16 color palette throughout.

It's a 8MB ROM, all but about 64KB or something is used. I didn't want to mess with screwy Ex ROM mappings so I picked S-DD1 to be my mapper. If I knew enough about it to make a codec I definitely would've, I could've bumped the resolution up to about double and maintain the framerate if ROM size permitted..bsnes source make it clear how $48xx worked and it was a hirom friendly chip so integrating it was no problem. Aslong as an emulator can play Star Ocean it should play this.

ZSNES doesn't work at all, it seems to look at the 8MB filesize and dismiss it instantly. snes9x works but I had to make it spool a byte more audio per frame or it crackled and looped incorrectly. bsnes works fine with the latest I just grabbed from byuu.org and a slightly older debugger. snes9x also seems to drift slightly with the audio sync but bsnes didn't, so I left that alone.

snes homebrew seems rather dead so here you go, I made a clunkier version before with a smw ad but it couldn't manage 60FPS without a bit of extra work on the source.

User avatar
tokumaru
Posts: 12050
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Tue Nov 24, 2009 10:03 pm

Very cool! I would very much like to see a video of something that's not a game though (preferably ripped from a high quality source, not a VHS or YouTube). Are you taking requests? =)

naI
Posts: 114
Joined: Fri Jun 26, 2009 4:58 pm

Post by naI » Tue Nov 24, 2009 10:42 pm

:shock: That's amazing! :D Is this all uncompressed video?

Now you can amaze your friends when they see you 'playing' a Nintendo 64 game on a Super Nintendo. :wink:

smkd
Posts: 101
Joined: Sun Apr 22, 2007 6:07 am

Post by smkd » Wed Nov 25, 2009 12:09 am

@tokumaru: yeah I can do requests, they're easy enough to throw together. Aslong as I can open the video properly in virtual dub (since adobe flash gave me hell with anything but flv files) then it's all good. Ofcourse if the source video is 30FPS or something then almost double the length is possible, 20FPS a bit under tripple etc. I'm not sure that with such low res that a YT vid will look all that bad, the reason I recorded the OP video myself is that they don't host 60FPS video.

@naI: It's all uncompressed pretty much, the code just splits a byte containing two 4bit pixels into two bytes, sends it to a mode7 tilemap after that. Mess around with the d-pad for generic effects, I didn't put anything fancy in though. haha yeah I'd like to throw it on a SNES cart if I could just for kicks, although realistically the only way that'd ever happen is if SNES powerpak had the MMC portion of the special chip, or there's a widely adopted method for getting ridiculously large ROMs that actually work in emulators + a real SNES that I don't know of.

I'd like to improve this but I'm not sure what to do. After all that optimizing, the processor sits on its ass doing nothing for nearly half a frame, but if I used that time for higher res video at larger frame rates I'd have like 30 seconds of playback lol.

User avatar
tokumaru
Posts: 12050
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Wed Nov 25, 2009 12:32 am

smkd wrote:@tokumaru: yeah I can do requests, they're easy enough to throw together.
I imagine that real life stuff wouldn't look very good because of all the colors, there would be too much dithering that doesn't work very well in such a low resolution. Maybe you could try a cartoon or an anime, I guess that would work out well. For anime they usually use 24fps, so you could fit a lot more, maybe an entire opening. 60 fps is overkill IMO, only games run at that speed, all other sources should look good with 30 fps or even less.
I'd like to improve this but I'm not sure what to do. After all that optimizing, the processor sits on its ass doing nothing for nearly half a frame, but if I used that time for higher res video at larger frame rates I'd have like 30 seconds of playback lol.
You could try adding some compression, nothing too sophisticated... Since you're not using any, even RLE should allow for a bit more video in there.

User avatar
MottZilla
Posts: 2835
Joined: Wed Dec 06, 2006 8:18 pm

Post by MottZilla » Wed Nov 25, 2009 12:43 am

That's a pretty impressive project. It would be interesting to see what else you can do with it.

MatthewCallis
Posts: 82
Joined: Sat Sep 22, 2007 8:32 am
Location: Seattle, WA
Contact:

Post by MatthewCallis » Wed Nov 25, 2009 10:04 am

It'd be awesome to see what you could fit into a movie for the PowerPak, you'd have to undo the SDD-1 like the Star Ocean hack but it would be fun to see.

smkd
Posts: 101
Joined: Sun Apr 22, 2007 6:07 am

Post by smkd » Wed Nov 25, 2009 10:24 am

@tokumaru: yeah a cartoon is a good idea, i just shuffled a bunch of things around and double buffer in the mode7 tilemap. Since the NES TMNT demo featured the song and I always liked that one, I did the intro video. 30FPS, with increased resolution I liked how it turned out. Might've looked better without the dither though, I don't know.

http://smkdan.eludevisibility.org/vidtmnt.zip

RLE would help, thing is the routine is very sensitive to anything you add to it. It's an unrolled loop repeated about 1800 times, add even a few cycles and its alot of extra effort. Even something simple like RLE has to be carefully put in or else performance goes to shit. Then again, at 20FPS or something, 2.5 full frame time is provided extra for all of this. Something is definitely workable here, there's just a ton of ways to do it. Will think it over, ofcourse if anyone has suggestions than go ahead.

I want to release the source but its pretty offensive to look at right now, just want to make it a little less ugly before uploading all of that.

Matthew: I read it has 16Mbytes of RAM but I wonder how its mapped to the SNES. Custom mappings would be neat, where you specify your imaginary PCB layout and it maps accordingly, definitely better than my S-DD1 hack.

User avatar
tokumaru
Posts: 12050
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Wed Nov 25, 2009 11:09 am

Coo! The sound is very good, and most of the times it's easy to tell what's going on in the video.

User avatar
MottZilla
Posts: 2835
Joined: Wed Dec 06, 2006 8:18 pm

Post by MottZilla » Wed Nov 25, 2009 12:52 pm

Wow! The TMNT video turned out really well I think. Never thought I'd see and hear the TMNT intro on SNES (well BSNES actually).

User avatar
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Post by Near » Wed Nov 25, 2009 2:59 pm

This has pretty much been a dream of mine for a long time. I loved your SMW commercial one, too.

I have many ideas to greatly improve upon this:

1. ROM size: I believe I cap the S-DD1 mapper at 8MB, but it would be trivial to allow up to 256MB ROMs. I'm happy to do this if you'd take advantage of it :)

2. Data size: since you are using the S-DD1 chip anyway, we can compress each video frame using the S-DD1 coprocessor, and you can stream DMA decompress that into video RAM. That should allow you to fit almost twice the amount of video data into the ROM, with no CPU overhead whatsoever.
Alternative: we may also be able to use the SPC7110's compression and MMC in the same way. Whichever works better, I suppose.

3. Use Mode 3: this will give you 256-color video, so you can make it look really sharp, and it's in tile format. This will make S-DD1 transfers possible via direct DMA, and you won't have to mess with interleaved tilemap stuff. Just set it up once and be done with it. Yes, this will "letterbox" the game, but I think it's okay. Better than overly pixellated with Mode 7.

4. Use PAL mode: this cuts the video rate from 60hz to 50hz, and greatly increases your Vblank time from lines 225-261 to 225-311.

5. Use video banking and halve the frame rate. Again, Mode 3 makes this possible. You can transfer half of the video frame on one frame, and the other half on the second. After the second frame, switch the VRAM base pointer between $0000 and $8000. There won't be any visible tearing by this method and you've doubled the video data you can transfer, eg doubled your possible output size.

6. Use part of the "active" display area. Another advantage to Mode 3 over Mode 7. Since the screen is letterboxed, you can take the height of your video, center it in the screen, and use HDMA to turn the screen off. For instance, if your height is 128, then turn the screen off from scanlines 0-63, and then from 192-311. Now you have even more time, and you can send even more video data.

7. Focus on widescreen instead of height. This will let you fit more video data since you have more scanlines with the screen turned off.

8. Again, use Mode 3. Create an RGB332 palette and store that into CGRAM one time. Now transcode your video to this format, and you no longer have to transfer any palette data ever.

9. If you do all of the above, and you get a really high quality video playing; I will work with you to take advantage of the 32KHz 16-bit PCM streaming via the SNES hardware's cartridge connector pins. I have already emulated these pins to allow Super Game Boy audio output to mix with the SNES' internal audio.
We can make a pseudo-special-chip that lets you stream raw PCM data (or ADPCM data) through these pins, giving you up to redbook quality audio, without requiring any CPU time. This will give us even more time to send more video data.

9a. I've also considered creating a pseudo-CD player base unit. Send the track commands to $21xx and it will start "streaming" via the base units' audio mixer pins the red book audio. The idea was for ROM hackers to then go and patch out song playback commands on the S-SMP to instead start playing from redbook tracks. This would allow a days' work to implement full orchestral soundtracks into existing SNES games like Chrono Trigger, Final Fantasy 6, etc. The same concept could be used here.

I'm very confident that we could push something like 192x144 video at 25fps in RGB332 with redbook audio in realtime.

Again, just let me know if you're interested in making it happen; and I'll do what I can to help :D

magno
Posts: 193
Joined: Tue Aug 15, 2006 5:23 am
Location: Spain
Contact:

Post by magno » Wed Nov 25, 2009 3:27 pm

Astonishing post, byuu!! A incredible practical application of SNES modes and resources :D

BTW, what is "redbook"?

User avatar
Near
Founder of higan project
Posts: 1553
Joined: Mon Mar 27, 2006 5:23 pm

Post by Near » Wed Nov 25, 2009 3:40 pm

Redbook is what CD audio tracks use.

Anyway, I did some proof calculations again: we can do 240x176x256color@25hz in real time.

240*176=42,240 bytes/frame

Scanlines are 1364 clocks long, and 40 are used by DRAM refresh. DMA eats 8 clocks per byte.

We double our bandwidth (and halve our framerate) using VRAM bank switching.

312-176=136

136*1324*2=360,128
360,128/8=45,016 bytes can be transferred per frame

We could push it farther, but I choose not to. That gives us 2,776 clocks to initiate the DMA transfers and swap the VRAM pointer every other frame. Similarly, we can reduce to 240x160 and gain a lot of additional clocks if we need more processing time.

Don't exceed 240 for the width, it will just be cut off by overscan anyway.

Now, bandwidth. I won't lie, it's massive.
At 42,240 bytes * 25 = 1,056,000 bytes per second. Let's round to 1MB/sec. We can cut that in half with the S-DD1, to reach 512KB/sec. That's roughly ~30MB/minute, so we'd have about eight minutes worth of TV-quality video at 256MB.

There are lots of options from here to increase that: utilize the SA-1 to compress data even more, fall back on 16-color mode (at which point we could easily do full 256x224 video), lower the output resolution, halve the framerate again to 12.5fps (really not that bad for animation), raise the mapper range to 512MB or even 1GB, detect "still" sections and omit their video data, detect "identical" frames (eg mouth moving is usually only two alternating frames) and re-use existing data, etc.

-----

This is about the quality we could expect, though perhaps a bit higher by using a better 24-bit -> 8-bit downsampler and a lossless source:
Image

tepples
Posts: 22332
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Post by tepples » Wed Nov 25, 2009 3:56 pm

smkd wrote:Aslong as I can open the video properly in virtual dub (since adobe flash gave me hell with anything but flv files) then it's all good.
And if not, you can always use ffmpeg to convert things to a Huffyuv or Xvid AVI, which Vdub will have no problems opening.
byuu wrote:Use PAL mode: this cuts the video rate from 60hz to 50hz, and greatly increases your Vblank time from lines 225-261 to 225-311.
But if you make something exclusively for the European market, don't you have to have English, Spanish, French, and German dubs?

240x160 could work. That's the much res GBA Video uses, but then GBA Video uses what would be a "special chip" (ARM7TDMI clocked at 16.8 MHz) compared to the Super NES.

User avatar
tokumaru
Posts: 12050
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Post by tokumaru » Wed Nov 25, 2009 4:13 pm

byuu wrote:4. Use PAL mode
I thought I'd express my objection to this particular suggestion. I'm really against PAL-exclusive games/demos, because that excludes a huge deal of users.

Post Reply