chr sheet of most used tiles

A place for your artistic side. Discuss techniques and tools for pixel art on the NES, GBC, or similar platforms.

Moderator: Moderators

JRoatch
Formerly 43110
Posts: 388
Joined: Wed Feb 05, 2014 7:01 am
Location: us-east
Contact:

chr sheet of most used tiles

Post by JRoatch » Mon Sep 11, 2017 10:14 am

So for no particular reason I thought it would be interesting to see what the most common tiles in all* games with CHR-ROM.

*: From a particular 2011 "nointro" ROM collection that was able to be parsed by "ines.py"

After some basic de-duplication of full sheets, I ran them through a script to count the frequency. The attached image shows 512 tiles starting from the from most occurring
2017-09-10_most-used-tiles.chr.png
2017-09-10_most-used-tiles.chr.png (2.71 KiB) Viewed 5952 times
Most of the tile are basic rectangles, shapes, letters from a common NES font, and X marks.

4 tiles popped up that seems to me should be unique, but appeared at least 300 times across my data set.
2017-09-10_intresting.png
2017-09-10_intresting.png (250 Bytes) Viewed 5952 times

User avatar
nesrocks
Posts: 434
Joined: Thu Aug 13, 2015 4:40 pm
Location: Rio de Janeiro - Brazil
Contact:

Re: chr sheet of most used tiles

Post by nesrocks » Mon Sep 11, 2017 10:17 am

Interesting research! But I think it should be once per game. Some games just repeat tiles in unused spaces, so I think it would be more interesting to see which tiles appear in most games, not counting how many times they appear in any particular game.
https://twitter.com/bitinkstudios <- Follow me on twitter! Thanks!

JRoatch
Formerly 43110
Posts: 388
Joined: Wed Feb 05, 2014 7:01 am
Location: us-east
Contact:

Re: chr sheet of most used tiles

Post by JRoatch » Mon Sep 11, 2017 11:37 am

2017-09-11_most-shared-tiles.chr.png
2017-09-11_most-shared-tiles.chr.png (2.1 KiB) Viewed 5932 times
OK, That did cut out the appearances of the "unused tile" graphics. Now it's pretty much only basic monochrome shapes.
Each of these tiles appears in at least 5% of the games, with the top four being in about %97 of the games

lidnariq
Posts: 8786
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: chr sheet of most used tiles

Post by lidnariq » Mon Sep 11, 2017 12:08 pm

When I went and made the fingerprint of 128-byte slices in GoodNES, I was surprised at just how many times the slice that was the numbers "01234567" (in the same font) showed up.

In hindsight, I probably shouldn't have been.

User avatar
nesrocks
Posts: 434
Joined: Thu Aug 13, 2015 4:40 pm
Location: Rio de Janeiro - Brazil
Contact:

Re: chr sheet of most used tiles

Post by nesrocks » Mon Sep 11, 2017 12:10 pm

Funny how many basic shapes can exist. Now someone take this sheet and make the most average NES game ever. :)
https://twitter.com/bitinkstudios <- Follow me on twitter! Thanks!

User avatar
Myask
Posts: 965
Joined: Sat Jul 12, 2014 3:04 pm

Re: chr sheet of most used tiles

Post by Myask » Mon Sep 11, 2017 1:45 pm

Analyzing what font characters are present…

Code: Select all

   ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890 (for alignment/comparison)
01 ABCDEF H     NOP   TU        45      c,©,OK
nothing on color 02
03 AB DEF HI    NOP   TU    Z123456789  2B

█ in all four colors wins, natch
▒ in 10, 01, 21, 12, 03, 30, 32; all of which are sensible ones. Odd that 32 isn't present.
We can see that people use colors in 0312 frequency, mostly.
Oddly, that second sheet now admits © in color 3 when it wasn't there before, and about halfway up the ranking.

Interestingly, we don't have all of ETAOINsHrDlU, and I/C aren't in both colors! Part of this is, I'm sure, the various nature of renderings of S. L doesn't make sense, though. Also interesting that no separate 0 rendering came up, though again, various renderings to distinguish from O…and colliding/sharing with O in some cases.

I am amused that 10 of 5-16 of Shared are what almost certainly are sprite-0 triggers.

Could you cull those one-rectangle on other-color shapes? At least, the ones where the rectangle touches three edges. (Noting as numbers these shapes, like "left 7/8ths 1 on 0", might be more readable than in a character sheet.)
Could you make a version where it treats palette swaps as additional instances of the first tile encountered with that pattern? Possibly allowing BG to be swapped, possibly not; for sprites it means something, after all.

Of course, this simply misses all CHR-RAM games.

tepples
Posts: 21752
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: chr sheet of most used tiles

Post by tepples » Mon Sep 11, 2017 1:46 pm

And given unalignment and compression, I don't think there's a good way to reach CHR RAM games.

User avatar
Myask
Posts: 965
Joined: Sat Jul 12, 2014 3:04 pm

Re: chr sheet of most used tiles

Post by Myask » Mon Sep 11, 2017 1:54 pm

Nothing as fast, surely. Running TAS or other game-completing keyrecordings through games on a CHR-logging emulator would be a naïve way to reach them, but, of course, is numerous magnitudes slower than what was done here.

If you wanted a less thorough scrape in that manner, perhaps, a wait for 1 minute, autofire a+start for a minute nputfile would probably get you through nigh-all Title, Menu, Pause, Game allocations. [waiting a minute as attract modes will be smörgåsbords!]

User avatar
Myask
Posts: 965
Joined: Sat Jul 12, 2014 3:04 pm

Re: chr sheet of most used tiles

Post by Myask » Mon Sep 11, 2017 2:56 pm

Rectangle heuristic:
2-color [if one color, well, that's already a rectangle, no need for further testing…]
has at most two different 8-pixel slices, which both fit the pattern and the distribution of which fit one of the nine A*8-0, B*0-8 (0, 1, 3, 7, F, 1F, 3F, 7F, FF; the complementary FE, FC, F8, F0, E0, C0, 80 should be taken care of by the palette-swap logic)

lidnariq
Posts: 8786
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: chr sheet of most used tiles

Post by lidnariq » Mon Sep 11, 2017 3:01 pm

I wonder how practical it would be to search with deduplication of bitplane permutations...?

ccovell
Posts: 1006
Joined: Sun Mar 19, 2006 9:44 pm
Location: Japan
Contact:

Re: chr sheet of most used tiles

Post by ccovell » Mon Sep 11, 2017 4:14 pm

This is really interesting!

Of course, that "X" and other tiles around it are for marking unused/empty tiles. The kanji there literally means "empty".

JRoatch
Formerly 43110
Posts: 388
Joined: Wed Feb 05, 2014 7:01 am
Location: us-east
Contact:

Re: chr sheet of most used tiles

Post by JRoatch » Mon Sep 11, 2017 4:43 pm

I tried out normalizing bitplane permutations. rectangles combined as a side effect. Attached is the count script I used for this.
2017-09-11_bitplanes-normalized.png
2017-09-11_bitplanes-normalized.png (3.79 KiB) Viewed 5836 times
If some of the letters look odd with a gray background, it's because the normalization sets the top left pixel to be always color 0.

I might manually search beyond 512 tiles for the whole alphabet, but that'll require using my eyes and may take a bit of time.

In the meanwhile here's a slightly more readable version of above.
2017-09-11_readable.png
2017-09-11_readable.png (4.4 KiB) Viewed 5830 times
ccovell wrote:The kanji there literally means "empty".
Ah, that makes sense. Thanks for the tidbit.

Edit: Former attachment is now inlined here:

Code: Select all

#!/usr/bin/env python3

import sys
import collections
import functools

all_tiles = collections.Counter()

@functools.lru_cache(maxsize=None, typed=False)
def normalize_tile(input_tile):
    # not the most efficient way, but it works.
    color_0 = bytes(((0xff^a) & (0xff^b)) for a, b in zip(input_tile[0:8], input_tile[8:16]))
    color_1 = bytes((a & (0xff^b)) for a, b in zip(input_tile[0:8], input_tile[8:16]))
    color_2 = bytes(((0xff^a) & b) for a, b in zip(input_tile[0:8], input_tile[8:16]))
    color_3 = bytes((a & b) for a, b in zip(input_tile[0:8], input_tile[8:16]))
    norm = sorted([color_0, color_1, color_2, color_3])
    norm.reverse()
    plane_0 = bytes((a | b) for a, b in zip(norm[1], norm[3]))
    plane_1 = bytes((a | b) for a, b in zip(norm[2], norm[3]))
    return plane_0 + plane_1

for input_filename in sys.argv[1:]:
    input_chr = open(input_filename, 'rb').read()
    #all_tiles.update(input_chr[i*16:i*16+16] for i in range(len(input_chr)//16))
    tileset = set(normalize_tile(input_chr[i*16:i*16+16]) for i in range(len(input_chr)//16))
    all_tiles.update(tileset)

with open("result.chr",'wb') as output_file:
    for tile in all_tiles.most_common(512):
        output_file.write(tile[0])
        print(tile[1])
Last edited by JRoatch on Fri May 04, 2018 11:31 pm, edited 1 time in total.

User avatar
Alp
Posts: 210
Joined: Mon Oct 06, 2014 12:37 am

Re: chr sheet of most used tiles

Post by Alp » Mon Sep 11, 2017 6:50 pm

JRoatch wrote: In the meanwhile here's a slightly more readable version of above.
The attachment 2017-09-11_readable.png is no longer available
Oh, now this is interesting!

I can't help but think that if this was organized, it would heavily resemble the Intellivision's built-in GROM.
intvfont.png
intvfont.png (2.25 KiB) Viewed 5807 times

User avatar
Bregalad
Posts: 7767
Joined: Fri Nov 12, 2004 2:49 pm
Location: Chexbres, VD, Switzerland

Re: chr sheet of most used tiles

Post by Bregalad » Tue Sep 12, 2017 12:12 am

Note that this is the most present tiles in the CHR-ROM not the most used, i.e. you ignore whether the games actually uses these tiles a lot, a little, or not at all, during gameplay. This is a major difference.
I tried out normalizing bitplane permutations.
Did you only account for bitplane permutations (exchanging colours 1 and 2) or any colour permutation ? It seems you did the latter, which makes more sense.

qalle
Posts: 50
Joined: Wed Aug 16, 2017 12:15 am

Re: chr sheet of most used tiles

Post by qalle » Sun Oct 29, 2017 9:02 pm

I extracted all unique 1- or 2-color tiles from 7080 NES games. There were about 800,000 such tiles. I normalized the colors within each tile: the more common color is always black and the less common color is white. (If there were 32 pixels of each, the colors were inverted if the top left pixel was white.)

I have attached an image of the tiles and the scripts I used. (Run at your own risk.)

Next, maybe try sorting the tiles by number of white pixels...
Attachments

[The extension py has been deactivated and can no longer be displayed.]

[The extension py has been deactivated and can no longer be displayed.]

[The extension py has been deactivated and can no longer be displayed.]

All unique 1- or 2-color tiles in 7080 NES games. Warning: over 50 million pixels.
All unique 1- or 2-color tiles in 7080 NES games. Warning: over 50 million pixels.

Post Reply