Idea for cell-shaded FMV on the NES

A place for your artistic side. Discuss techniques and tools for pixel art on the NES, GBC, or similar platforms.

Moderator: Moderators

User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Idea for cell-shaded FMV on the NES

Post by tokumaru »

I recently watched some cool anime-style intro sequences on old games, like Astal, Silhouette Mirage and even Sonic CD (one of my favorite intro sequences since forever), and started thinking of a way to do something remotely similar to that on the NES.

Due to the limited memory and PPU bandwidth, the first thing I decided is that the pictures would be created with NT/AT updates only, instead of expensive pattern table manipulations, which are also impossible with CHR-ROM. Keeping things compatible with all the different mappers and types of CHR memory meant using only 512 tiles all the way through. The problem is that packing color combinations in so few tiles usually results in extremely chunky pixels.

Being impressed at how decent rgb121 alternating color components every scanline looked, I started thinking of how to do something similar. Based on the fact that many video codecs encode lightness with more resolution than color, due to the fact that the human eye is more sensitive to lightness, I decided to make that same separation. At 2 levels of brightness, I can pack 8 pixels in a tile, for a total of 256 combinations. If I arrange them in a 4x2 pattern, I can get a monochrome 128x60-pixel picture out of a single name table. As for the colors, 4 colors arranged in a 2x2 pattern (using the remaining 256 tiles) can represent a 64x30-pixel image.

Basically, the name tables would look like this (please don't mind the image I used for testing, it's just some cartoon looking thing I found on Google):
fmv-nt.png
fmv-nt.png (1.85 KiB) Viewed 7600 times
The interesting thing about having the color image use only half of its name table is that at least that part of the image can be double buffered, reducing the amount of bytes to update at a time every time the picture changes. Anyway, since the vertical resolution is not that good and the display will need constant scroll manipulation and pattern table switching, we need borders. The most straightforward thing to do is to squeeze the image to half of its height. Unfortunately that means that half of the screen is occupied by borders, and the video ends up looking very, very wide. This invokes a certain cinematic vibe so I don't think it's so terrible. Anyway, here's the result after combining the two pictures:
fmv-still.png
fmv-still.png (1.18 KiB) Viewed 7600 times
Yes, it's stripy and washed out, but it looks much more detailed than the color image alone could. The higher resolution lightness image adds a lot of detail, and allows for light and dark variations of each color in the color image.

Since one palette is used for the lightness image, 3 are free for the color image. Each 4x2-pixel area (area affected by a pair of attribute bits) in that image can have only 4 colors, and the 3 palettes must share 1 color (color 0), because of how the PPU works. Color 0 is not hardcoded to black (or any other color), which means that the palette has to be changed mid-frame because of the border. There is one free color in the lightness palette that could be permanently set to black and pointed to with rendering disabled, but in order to update the name table for the next frame the PPU address register will have to point away from it at some point, so a palette change would be needed anyway.

The frame rate would be only 12fps (which is fairly standard for hand-drawn animation, although they frequently use motion and compositing to make it look more fluid), to save memory and provide more time for processing each image. Here's a little test featuring scaling and rotation (I didn't have time to draw/rip a proper animation):
fmv-test.gif
fmv-test.gif (66.24 KiB) Viewed 7600 times
Obviously, images have to be drawn with this specific format in mind. If you simply take an existing video and try to convert it to this format it will surely look like crap. Ideal images should have thick outlines, few colors in localized areas, and very simplistic backgrounds. Zoomed out scenes should also be avoided, in favor of close-up shots.

I haven't given much thought to the final (and really important) step, which is compression. Each uncompressed frame is nearly 1.5 KB, so there's definitely room for improvement.

Any comments, suggestions, opinions?

P.S.: Bregalad, I won't be offended if you say it looks like crap. :lol:
Sik
Posts: 1589
Joined: Thu Aug 12, 2010 3:43 am

Re: Idea for cell-shaded FMV on the NES

Post by Sik »

You can remove a few tiles at the sides (they will not be visible anyway due to overscan), this will give you a bit more of room to play with.

Also if you can afford it, consider alternating the position of the scanlines (60FPS blinking, yeah), that will reduce the strippiness (and before somebody comments on seizures, strips also cause them anyway, if anything this may make it less prone to cause them due to being harder to see).
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Idea for cell-shaded FMV on the NES

Post by tokumaru »

Sik wrote:You can remove a few tiles at the sides (they will not be visible anyway due to overscan), this will give you a bit more of room to play with.
Interesting suggestion, but how would I arrange them vertically in the final picture?
Also if you can afford it, consider alternating the position of the scanlines (60FPS blinking, yeah), that will reduce the strippiness
That's something I planned for initially, and is totally doable, but the tests didn't look good (when viewed in a computer screen). It was really jumpy, and I kinda think that the stripy look is less blocky.

BTW, the workflow for creating the animations doesn't have to be much different from what Flash animators are used to. I draw the objects and characters in Illustrator (using a palette containing only the washed out colors) and export high-resolution images with no anti-aliasing. Then I import the images in Flash and disable smoothing for them. Then I just use Flash normally, trying to respect the limitations of the video format I explained, and I export the result as a high-resolution image sequence, ready to be processed by the converter. Avoiding anti-aliasing is important to make sure the encoder will always pick the correct colors.
User avatar
Bregalad
Posts: 8055
Joined: Fri Nov 12, 2004 2:49 pm
Location: Divonne-les-bains, France

Re: Idea for cell-shaded FMV on the NES

Post by Bregalad »

This could potentially look great, but I feel like your demo using a badly drawn teddy bear doesn't do it justice.

I agree that limiting to 512 tiles and only NT/AT updates is a good idea. I however did not understand how you got arround the colour limitation. Also how'd you store the FMV in a format that is reasonably efficient in ROM ?
Sik
Posts: 1589
Joined: Thu Aug 12, 2010 3:43 am

Re: Idea for cell-shaded FMV on the NES

Post by Sik »

tokumaru wrote:Interesting suggestion, but how would I arrange them vertically in the final picture?
Just leave them blank? The idea is to not have to store them in the stream (or possibly rearrange some of them elsewhere if that's affordable).
User avatar
za909
Posts: 248
Joined: Fri Jan 24, 2014 9:05 am
Location: Mijn hart woont al in Nederland

Re: Idea for cell-shaded FMV on the NES

Post by za909 »

This looks really promising, and way more advanced than how the Bad Apple demo looked.
If the space becomes a huge issue I think an oversized BNROM comes to mind or something like that...
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Idea for cell-shaded FMV on the NES

Post by tokumaru »

Bregalad wrote:This could potentially look great, but I feel like your demo using a badly drawn teddy bear doesn't do it justice.
True, but I wanted to see how the artifacts would look over time and that's the closest thing to the specifications that popped up on Google. I'll try to find the time to draw something myself.
I however did not understand how you got arround the colour limitation.
What limitation are you talking about, specifically? In the end, these are just 2 pictures, one occupying a full name table (lightness image) and another occupying half of a name table (color image). The lightness image uses only 1 palette (or rather, 2 entries of that palette), and the remaining 3 palettes are distributed as necessary in the color image. If you spread the objects/characters nicely around the picture, avoiding having to repeat colors, you could get the impression of having 20 colors on screen ((3 colors x 3 palettes + 1 shared color) x 2 shades).
Also how'd you store the FMV in a format that is reasonably efficient in ROM ?
That's the part I haven't given much thought to yet. There should definitely be some temporal delta coding, but I haven't though of the specifics yet. I'm open to suggestions.
Sik wrote:Just leave them blank? The idea is to not have to store them in the stream (or possibly rearrange some of them elsewhere if that's affordable).
Ah, so you're were just suggesting a way to reduce memory usage... I though you suggested that as a way to increase vertical resolution. Yeah, I guess I can cut a few bytes off that way.
za909 wrote:If the space becomes a huge issue I think an oversized BNROM comes to mind or something like that...
I definitely want to think of a good compression scheme for this. I think the goal here is to have a 30 second intro (360 frames) use 128KB or so. That's about 364 bytes per frame, which may sound like a lot coming from 1.5KB, but with good usage of the temporal redundancy I think it might be possible.
User avatar
dougeff
Posts: 3078
Joined: Fri May 08, 2015 7:17 pm

Re: Idea for cell-shaded FMV on the NES

Post by dougeff »

I like it.

Just to be clear...one name table is (2 colors) black and white, and one name table is black, white, brown, and pink (in this example)? And you're limiting the first one to (2 colors) black and white to reduce data size?

Well, any FMV that works on a real NES would be great. It would still be nice to add music or real audio samples longer than 1 second long.
nesdoug.com -- blog/tutorial on programming for the NES
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Idea for cell-shaded FMV on the NES

Post by tokumaru »

dougeff wrote:one name table is (2 colors) black and white
It's black and gray, actually. I tried white but it didn't blend as well with the colors. Initially I tried dark gray and light gray, but the NES is really lacking in grays (if you don't use the grays that don't exist in the RGB PPUs), and black improved the contrast a bit.
and one name table is black, white, brown, and pink (in this example)?
Yes, but there's a second palette being used for the bracelet that has yellow instead of pink. Normally you'd avoid this kind of thing in order to not waste palettes with lots of repeated colors. Such details would be dropped (yellow would become pink or white) in favor of a more varied palette that could be used elsewhere if necessary.
And you're limiting the first one to (2 colors) black and white to reduce data size?
It's actually to be able to draw 8 pixels per tile and still stay under the limit of 256 tiles. In order to have more colors, the resolution would have to be reduced which is exactly what I tried to avoid.
Last edited by tokumaru on Sun Aug 23, 2015 12:49 pm, edited 1 time in total.
User avatar
Bregalad
Posts: 8055
Joined: Fri Nov 12, 2004 2:49 pm
Location: Divonne-les-bains, France

Re: Idea for cell-shaded FMV on the NES

Post by Bregalad »

What I don't understand is how do you merge both colours and black/white layers into the FMV? Do you alternate between them every scanline? As long as there is no flickering I'm happy with that :)
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Idea for cell-shaded FMV on the NES

Post by tokumaru »

Bregalad wrote:Do you alternate between them every scanline?
Exactly. At small sizes, like in the images I posted, the colors blend so well you can hardly tell. Zoom in and you'll see. It will definitely not blend as well in full screen, but the overall result is still acceptable IMO.
As long as there is no flickering I'm happy with that :)
Flickering indeed looked bad in my tests, but I'm willing to see how it looks on the real console.
tepples
Posts: 22705
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Idea for cell-shaded FMV on the NES

Post by tepples »

Sik wrote:You can remove a few tiles at the sides (they will not be visible anyway due to overscan)
Side tiles are very visible on every recent TV that I've used. True, the official background planning sheets used 224x192 (which corresponds to 10% border per side), but modern video standards are closer to 3-5% border per side. The 4:3 frame in NTSC is 280x240 NES pixels in size, and a (280 - 256)/2 = 12 pixel border on the left and right sides is 4.2%.
Sik
Posts: 1589
Joined: Thu Aug 12, 2010 3:43 am

Re: Idea for cell-shaded FMV on the NES

Post by Sik »

Many recent TVs will show even the border area, for that matter.

It was just a suggestion to reduce memory usage some more without it being really noticeable in practice.
User avatar
Dwedit
Posts: 4922
Joined: Fri Nov 19, 2004 7:35 pm
Contact:

Re: Idea for cell-shaded FMV on the NES

Post by Dwedit »

How about RLE with predictors? Might work well enough.
Here come the fortune cookies! Here come the fortune cookies! They're wearing paper hats!
User avatar
tokumaru
Posts: 12427
Joined: Sat Feb 12, 2005 9:43 pm
Location: Rio de Janeiro - Brazil

Re: Idea for cell-shaded FMV on the NES

Post by tokumaru »

You mean something like what's described in this article? I guess the prediction is also helpful for taking advantage of vertical redundancy, which regular RLE doesn't, right? Looks like it'd work well, at least for the color image.

The B/W image, which is responsible for most of the frame's size, looks highly compressible to me. I remember having some success with 1bpp images using quadtrees. I'll have to look into that.

Anyway, I'm hoping that the temporal redundancy will be of great help too. My idea is to keep one frame worth of data in RAM at all times, so I can draw a new frame on top of the previous one, by patching the differences. Since this frame would occupy around 1480 bytes of RAM, there would still be 568 or so bytes left (unless you're using extra RAM, obviously) to hold the state of the game and keep basic things like audio working. Sounds reasonable to me, considering that not much else should be running while videos are playing.
Post Reply