It is currently Mon Dec 10, 2018 5:34 pm

All times are UTC - 7 hours





Post new topic Reply to topic  [ 3 posts ] 
Author Message
PostPosted: Tue Sep 18, 2018 3:05 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 20851
Location: NE Indiana, USA (NTSC)
I just had a conversation with the maintainer of j0CC-FamiTracker in the FamiTracker users' Discord server. It turns out that the Android operating system relies on media types (also called MIME types), not filename extensions, for routing "intents" (structured command lines) to applications. I've discovered that the "Open With..." functionality of the Downloads list in Firefox for Linux also depends on media types, as a file served as application/octet-stream consistently opens in the wrong application.

So are there widely used media types that correspond to common filename extensions used by software targeting old Nintendo platforms?

These might be in application/*: .nes, .sfc, .gb/.gbc
These might be in audio/*: .nsf, .ftm/.0cc, .spc, .gbs

If not, it might be time to write some authoritative format docs and register the media types with IETF.


Top
 Profile  
 
PostPosted: Tue Sep 18, 2018 5:11 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3716
Location: Mountain View, CA
There is a mapping between MIME types (e.g. derpderp/ilikesnakes) and file extensions (e.g. .ils): it's usually called the mime.types file, and is usually provided by the OS, else with the individual application that cares about MIME types. This is a universal norm, regardless of OS (though Windows does it differently; it's crap in the registry, and also defines the association with what program(s) said thing should be opened with).

For webservers, this same model is used for static file serving, i.e. the webserver is configured so that http://derpderp/path/somefile.ils refers to /some/place/on/filesysystem/somefile.ils. The webserver needs to know what MIME type matches the .ils file extension. For Apache, mod_mime is used for this purpose -- see TypesConfig directive for a full description of how this works (I recommend reading this for general comprehension anyway). But this model/approach/method is not Apache-specific, it's used by nginx (see mime.types which is just something that's include'd), IIS, and many others.

But what if there isn't a match? The webserver can do other things, like do actual byte-level file examination (think *IX file command) as a fallback, and make a decision from that. I've used this with both success, and grief, throughout my lifetime -- every situation varies.

What if there still isn't a match? Then the webserver will fall back to whatever the default MIME type is that's configured, or what's set as an override (for example, Apache has ForceType). In some cases, there is no Content-Type header returned at all. But other times, that's text/plain, which is not always what's wanted (and of course, that may not get "decoded" correctly). Talk to your webmaster to solve whatever the problem is; for static content serving, solving this is easy as pie. For Today's WebCrapOSphereOfJunk, the problem is made unnecessarily complicated -- the back-end software that's responsible for returning said HTTP headers then has to try and do all of the above itself. Ever wonder why so many websites/things get this wrong and botch it? Now you know why.

As for things you want to be downloaded that also contain a filename: this is controlled by the HTTP (server) header Content-Disposition. It's a stupid header that uses a stupid syntax, but it's what we've got. The attachment part is relevant, but you'll enjoy the hilarious syntax regardless (you'll find a lot of people do very different things with this header syntax-wise. ;-) ).

For sake of example: something returning Content-Type: application/octet-stream **and** Content-Disposition of an attachment with a file extension, should cause the OS/distro/application to correlate the file extension with whatever actions are desired (could be "Save as...", auto-opening with something, or prompting you for any of those things). The fact it's application/octet-stream isn't entirely relevant to that decision (though it is important to ensure the content gets downloaded as raw binary and not mangled in other ways).

application/octet-stream is considered, per RFC 2046, a "binary file format" of nebulous nature. It's intended for pretty much anything that isn't "web-oriented", and I strongly believe it applies to all of these formats. I believe the IETF would agree.

Which brings me to this:

Since you're talking about sticking presumably .ftm files directly onto a Discord server: do Discord's servers actually return a Content-Disposition header for files of this sort? The answer: yes but it depends on what type of file you've uploaded to Discord.

For a .jpg file: https://cdn.discordapp.com/attachments/ ... tagram.jpg

curl -v results, with irrelevant headers removed:
Code:
< content-type: image/jpeg
< content-length: 63010

For a .zip file -- note the difference: https://cdn.discordapp.com/attachments/ ... 9/blah.zip

Code:
< content-type: application/zip
< content-length: 121
< content-disposition: attachment;%20filename=blah.zip

This should, ideally, pop up a "Save as..." dialog with the filename pre-populated. Unless you've configured your OS/distro/whatever to handle .zip differently, ex. auto-open some unzipper program. It can know how to do this based on the file extension.

For a .ftm file: https://cdn.discordapp.com/attachments/ ... nl_end.ftm

Code:
< content-type: application/octet-stream
< content-length: 9985
< content-disposition: attachment;%20filename=enl_end.ftm

Which should, in effect, behave similarly to .zip.

In other words: it looks to me like Discord's servers is handling things how I would expect them to. (Consider me surprised!)

What this really does is bring into question the client (Firefox, in this case), and/or your OS/distro/WM, and what behaviour *you* expect from it. Maybe it's deferring to your OS/distro/WM to decide what to do next. Maybe it isn't. Best ask Mozilla, see what their take on it is. It looks to me like someone has already complained about this similarly in the past (on a Linux machine nonetheless). The responses there explain to you how to tweak your window managers' stuff to get it to behave how you wish. Look at the file format of that file -- it isn't like mime.types from above, it's a different format that also includes extension. You're welcome!

You can send all Android complaints/concerns/whatever to Google. Said file formats are going to be too obscure for them to care -- really. You could open up that discussion with them though. Are you aware of any mobile phone apps that can play FTM files? I'm not.

My view regarding the IETF registration:

For ROM file extensions, this is a pointless endeavour -- really. You aren't going to find a standard for this no matter how hard you try. You ever seen games released as .bin? :-) Good luck, not going to happen.

For audio/whatever, I'm more accepting, but this shows a lack of understanding of what IETF registration actually involves, per RFC 6838. The requirements are both strict and loose at the same time. Going off of memory from when I took a job that was considering filing for one, these were some concerns I remember bring brought up:

- There generally has to be commercial and commonplace software to be associated with said media type/file format. Yes, I said commercial. Non-commercial things tend to fall under the prs MIME type, which stands for "personal"
- The file format involved must be sufficiently, and clearly, documented. Likewise, byte ordering concerns sometimes justify more than one registration, depending on what the format is,
- A security review of the file format. I don't think this is a requirement (it didn't used to be anyway), but it's relevant. Pretty sure semi-recently someone found a really obscure way to work through a music file format (I think of some retro console or computer? Can't remember) to run programs or run code on the system itself, and some mainstream libraries used in *IX had to be updated
- This might have changed, but: an RFC has to be submitted describing the whole format/media type/thing, not just some "random text file" (and a Wiki page will not suffice -- ever. RFCs are a lot more serious). If something is "personal" then I think this is excluded from the requirements, but you should check,
- A dedicated individual or organisation responsible for the IETF registration going forward. This individual/org is the only one who can update/amend/change the registration for that type. This is known as the "owner of the media type".

And these are just the ones I remember. I haven't looked at that RFC in years, so there's probably a lot more I've forgotten. I can't remember every RFC. Anyway, seems like a lot of work for very little gain.


Top
 Profile  
 
PostPosted: Tue Sep 18, 2018 5:56 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 20851
Location: NE Indiana, USA (NTSC)
koitsu wrote:
You can send all Android complaints/concerns/whatever to Google. Said file formats are going to be too obscure for them to care -- really. You could open up that discussion with them though. Are you aware of any mobile phone apps that can play FTM files? I'm not.

I'm aware of such an app that the maintainer of j0CC-FamiTracker is working on. It's intended to work by converting FTM to NSF and handing the NSF off to Droidsound-E or another installed NSF player. But the lack of a media type for FTM and NSF is causing problems with getting this to work.

koitsu wrote:
- The file format involved must be sufficiently, and clearly, documented. Likewise, byte ordering concerns sometimes justify more than one registration, depending on what the format is,

NSF is a 128 byte header with a described format followed by a block of bytecode almost* identical to MOS 6502 machine code, which calculates writes to MMIO registers that are also pretty well documented. I concede that I don't know the fine details of translating the docs on the wiki into RFC-ese.

koitsu wrote:
- A security review of the file format. I don't think this is a requirement (it didn't used to be anyway), but it's relevant. Pretty sure semi-recently someone found a really obscure way to work through a music file format (I think of some retro console or computer? Can't remember) to run programs or run code on the system itself, and some mainstream libraries used in *IX had to be updated

The vulnerability in gstreamer's gstnsf component was discussed in this topic. The takeaway is that all bank numbers need to be bounds-checked.

koitsu wrote:
This might have changed, but: an RFC has to be submitted describing the whole format/media type/thing, not just some "random text file" (and a Wiki page will not suffice -- ever. RFCs are a lot more serious). If something is "personal" then I think this is excluded from the requirements, but you should check,

Last I checked, an RFC was required for a top-level registration, and other forms of were acceptable in vnd or prs.

koitsu wrote:
- A dedicated individual or organisation responsible for the IETF registration going forward. This individual/org is the only one who can update/amend/change the registration for that type. This is known as the "owner of the media type".

This is probably the root of the problem. As Paul Molloy of Infinite NES Lives has pointed out a few times, there is as of yet no entity that represents the NES homebrew community as a whole.


* Except that ADC and SBC ignore the D bit of processor status


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 7 hours


Who is online

Users browsing this forum: Google [Bot], Memblers and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group