Please add .UNF extension to the list of allowed files

Found an issue with the phpBB system here at NESdev? Use this forum to report problems.

Moderator: Moderators

lidnariq
Posts: 9496
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Please add .UNF extension to the list of allowed files

Post by lidnariq » Thu May 03, 2018 4:10 pm

I'd worry about allowing html/xhtml permitting XSS attacks. (I don't know that it would, it's just the obvious failure mode)

User avatar
koitsu
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Please add .UNF extension to the list of allowed files

Post by koitsu » Fri May 04, 2018 5:43 pm

DO NOT allow html/xhtml/shtml/php/js/anything else along these lines. You make blind assumptions about the infrastructure of the systems used to serve the website. All it takes is one Content-Type (MIME type) header and suddenly something is being run vs. downloaded (and not necessarily by the client; see: reverse proxying). DO NOT DO IT. I shouldn't have to state stuff like this. The last thing you want is WhoaMan finding out there's been a security hole open for some time due to someone allowing a silly extension through.

People wanting to use the formats: archive/zip up your work and upload it. It's not hard (either through GUI or CLI). The end.

tepples
Posts: 22014
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Please add .UNF extension to the list of allowed files

Post by tepples » Fri May 04, 2018 6:14 pm

I'll take that as a second, koitsu. Thank you. And for that reason, I don't even feel safe adding .svg, which also has a <script> element.

I have made changes to the following categories of allowed extensions.
  • "Active content" that may accidentally be executed in the browser context, causing cross-site scripting:
    Removed .swf, .js
    Removed .html, .htm, .xml (because <script> element and on* attributes)
    Did not add .svg, .xhtml (because <script> element and on* attributes)
  • Common CGI languages that may accidentally be executed in the server context:
    Removed .php, .py, .lua
  • Compiled languages:
    Added .cs, .java
  • Chip music scores:
    Added .0cc, .ly, .pently

User avatar
koitsu
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Please add .UNF extension to the list of allowed files

Post by koitsu » Fri May 04, 2018 6:19 pm

Changes look sound to me. Thumbs up.

User avatar
rainwarrior
Posts: 7822
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Please add .UNF extension to the list of allowed files

Post by rainwarrior » Fri May 04, 2018 6:26 pm

koitsu wrote:I shouldn't have to state stuff like this.
Ah, but who allowed .php in the first place? ;)
https://forums.nesdev.com/viewtopic.php ... 34#p127134

I'm a little disappointed to see .lua removed (there's been lots of cool FCEUX lua scripts shared in the past), and .py but whatever. More zips I guess. (I was the person who asked for both of those in the first place.)

Was either .js or .php ever actually requested? .swf?

tepples
Posts: 22014
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Please add .UNF extension to the list of allowed files

Post by tepples » Fri May 04, 2018 6:35 pm

You can still upload Lua scripts. Just zip them up first so that they don't accidentally get executed on the server.

User avatar
rainwarrior
Posts: 7822
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Please add .UNF extension to the list of allowed files

Post by rainwarrior » Fri May 04, 2018 7:06 pm

tepples wrote:You can still upload Lua scripts. Just zip them up first
Yes, that's what I was disappointed about. The friction of un-zipping propagates also to each person who wants to download it too.
tepples wrote:so that they don't accidentally get executed on the server.
I understand that part. Whatever you feel is necessary to protect the server is fine. I don't know anything about what your server's configuration looks like, so I'm in no position to tell you what's safe for the server, but as an end user I'm still disappointed that something I liked using (both up and down) is being removed.

Especially because this makes several old posts inaccessible, without even being shown a filename or any information to cross reference what might have appeared there with files I might still happen to have downloaded. It's a bit frustrating to have content you uploaded to the BBS for archival purposes suddenly effectively "deleted" with no identifying reference...

Like I would have a hard time finding my affected posts at this point, and then also knowing what content is actually missing is also a problem with this interface, and then even if I knew the filename I'd have to hope I still have a copy somewhere else that I can zip up and edit back into the post.

So... my disappointment is a bit more than "just" having to zip some files up in the future.


If you need to have them disabled for security reasons, I'm not trying to fight about that, you can weigh that as you need to, I'm just telling you how I feel about it as an end user, but is there anything you can do about old posts, at least? From my side I have no way of finding or recovering the now blocked content. That stuff is actually quite frequently useful to me. (Plus even if I had, e.g. a grace period and list of my own affected posts... that still doesn't work for anyone else's old posts who's not currently watching the issue and actively working to update with zips.)

User avatar
rainwarrior
Posts: 7822
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Please add .UNF extension to the list of allowed files

Post by rainwarrior » Fri May 04, 2018 7:28 pm

Just in case the problem isn't visible to moderators (tepples has mentioned interface differences in the past), this is what an attached lua file currently looks like for me:
lua_disabled_user_version.png
lua_disabled_user_version.png (2.69 KiB) Viewed 12691 times
Apparently for my own old posts, I can edit them and it will at least tell me the filename, but can't get back the content. (Attempting to download it will say it's blocked.) Edit: it seems I can get the filename for other peoples' posts by using the Quote button, but only if it was placed inline, I think.

User avatar
koitsu
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Please add .UNF extension to the list of allowed files

Post by koitsu » Fri May 04, 2018 8:49 pm

rainwarrior wrote:
koitsu wrote:I shouldn't have to state stuff like this.
Ah, but who allowed .php in the first place? ;)
https://forums.nesdev.com/viewtopic.php ... 34#p127134
I think the server infrastructure changed between then and now (including the webserver, IIRC; it used to be Apache, now it's nginx, and I think there's a reverse proxy involved now). What I knew to be true then I don't think is true now.

MIME types can be treacherous territory; server-side they seem innocent enough ("it's just a Content-Type header!"), but when reverse proxying is involved or potentially other devices like load balancers, all of which tend to inspect content, it becomes risky. Apache's mod_mime_magic can be a blessing and a curse too. Often feels that the days of basic web hosting/content serving are long gone. Things were simpler back then (code directly on an Apache webserver which was directly on the Internet, no intermediary anything).

Reviewing the download links from phpBB (example), we can see that the Content-Type returned (at least for a .zip) is application/octet-stream -- good -- and a Content-Disposition type of attachment-- which is correct and VERY important -- but the rest of that header looked bizarre to me (those are two apostrophes next to one another BTW, not a double-quote; the asterisk also made me go "?!?!"):

Code: Select all

$ curl -s -v 'http://forums.nesdev.com/download/file.php?id=10609'
*   Trying 208.71.141.55...
* TCP_NODELAY set
* Connected to forums.nesdev.com (208.71.141.55) port 80 (#0)
> GET /download/file.php?id=10609 HTTP/1.1
> Host: forums.nesdev.com
> User-Agent: curl/7.59.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Sat, 05 May 2018 03:28:01 GMT
< Content-Type: application/octet-stream
< Content-Length: 284
< Connection: keep-alive
< Keep-Alive: timeout=60
< X-Powered-By: PHP/5.5.9-1ubuntu4.20
< Set-Cookie: XXX
< Set-Cookie: XXX
< Set-Cookie: XXX
< Pragma: public
< Content-Disposition: attachment; filename*=UTF-8''700-in.1_32kib.zip
< Last-Modified: Tue, 31 Oct 2017 22:49:03 GMT
<
* Failed writing body (0 != 284)
* stopped the pause stream!
* Closing connection 0
For Content-Disposition, it looks like filename* is an RFC 5987 extension... from 2010, so no wonder I'm not familiar with it. Reading (well, I skimmed) that RFC, it looks as if the syntax is in fact correct. Learned something new. Though, it does make me wonder what happens if you upload a .txt that's in something other than ASCII or UTF-8, ex. JIS). I'd have to check.

Finally, client-side MIME type association is often a crap shoot as well -- you have no control over how someone's browser is set up/configured, so you don't know what will happen if the client receives a true/literal Content-Type that matches a MIME type that they've configured to allow to auto-run (e.g. "Download as..." vs "Open file"; scarily, a lot of people still do the latter, either automatic or manual). For example, we don't know if someone has .bat set to automatically run cmd.exe on it, and some jackass uploads one that does @echo off\rrmdir /q /s C:\WINDOWS. The idea is to minimise the chance of something like that happening. TMK, phpBB doesn't do any kind of "filtering" or "scanning of content" on uploads -- and I tend to fear stuff like that anyway (false positives causing failures that drive the uploader crazy).

These days, all it takes is an intermediary (ex. reverse proxy on the server side, and sometimes even a caching proxy on the client's network (semi-common at workplaces)) to cause a bit of mayhem with MIME types or filtering out certain headers (the latter is VERY common with reverse proxies). The time to worry is when Content-Type: application/octet-stream becomes, for example, Content-Type: application/javascript and there's no Content-Disposition header.

tepples
Posts: 22014
Joined: Sun Sep 19, 2004 11:12 pm
Location: NE Indiana, USA (NTSC)
Contact:

Re: Please add .UNF extension to the list of allowed files

Post by tepples » Fri May 04, 2018 9:16 pm

I wasn't aware that this feature of phpBB 3 removed previously uploaded attachments from view. As an administrator, I cannot download them either. In order to preserve both availability (no removal of legitimate downloads from public view) and integrity (no unexpected execution on the server), am I now expected to spend time writing a script that spiders the entire forum looking for attachments with extensions that have been deactivated, temporarily enable them, download them, use my moderator powers to upload them with the attachments properly zipped, and re-disable them?

User avatar
koitsu
Posts: 4218
Joined: Sun Sep 19, 2004 9:28 pm
Location: A world gone mad

Re: Please add .UNF extension to the list of allowed files

Post by koitsu » Fri May 04, 2018 10:12 pm

Hrm that's both a positive and a negative feature I'd say. Yeah, hrm. There can't be *that* many attachments of now-excluded extensions. Maybe you'd be better off doing it server-side and doing your best to go through the phpBB MySQL tables and see if you can pull out what you need from there. Might be quicker, and certainly less rude on the HTTP server.

lidnariq
Posts: 9496
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Please add .UNF extension to the list of allowed files

Post by lidnariq » Fri May 04, 2018 10:14 pm

rainwarrior wrote:It's a bit frustrating to have content you uploaded to the BBS for archival purposes suddenly effectively "deleted" with no identifying reference...
Out of idle curiousity (not anything actionable), do any of the python scripts you've uploaded show up in the Manage attachments list?

User avatar
rainwarrior
Posts: 7822
Joined: Sun Jan 22, 2012 12:03 pm
Location: Canada
Contact:

Re: Please add .UNF extension to the list of allowed files

Post by rainwarrior » Fri May 04, 2018 10:28 pm

lidnariq wrote:
rainwarrior wrote:It's a bit frustrating to have content you uploaded to the BBS for archival purposes suddenly effectively "deleted" with no identifying reference...
Out of idle curiousity (not anything actionable), do any of the python scripts you've uploaded show up in the Manage attachments list?
Ah, yes they do. At least there's a list of my own posts I can access then. (...and yeah, can see the filename and thread but can't download.) I thought I'd uploaded more lua scripts than python, but apparently it's the other way around.
tepples wrote:...am I now expected to (solve this problem)
You can decide how and whether to work on this. I'd volunteer to help, if I could, but I don't think I can really do much about it as a non-administrator. (If there is work I can do to facilitate this, though, let me know.)

I would suspect/hope that for most of them, the number of affected files is actually zero, but .py and .lua specifically are ones I'd been using and seen others using too. (It's possible this affects my posts more than anyone else's... I know I'm responsible for requesting these two formats in the first place.)

lidnariq
Posts: 9496
Joined: Sun Apr 13, 2008 11:12 am
Location: Seattle

Re: Please add .UNF extension to the list of allowed files

Post by lidnariq » Fri May 04, 2018 11:09 pm

For whatever it's worth, attachments with the forbidden extension return "404 Forbidden" in response to a HEAD request. (In contrast, "403 Forbidden" for PM attachments and "404 Not Found" for stuff that's actually gone).

With 12500-ish current attachments on the forum that's a little too big to just manually check without explicitly getting WhoaMan's OK.

Post Reply