It is currently Thu Oct 19, 2017 12:15 am

All times are UTC - 7 hours





Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 47 posts ]  Go to page Previous  1, 2, 3, 4  Next
Author Message
PostPosted: Thu Apr 10, 2014 10:10 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
I will try to make the time to undo this come the weekend. Remember: I have many, many wiki documents to manually edit by hand any time this thing is enabled or disabled.


Top
 Profile  
 
PostPosted: Fri Apr 11, 2014 2:10 am 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 2962
Location: Tampere, Finland
koitsu wrote:
I will try to make the time to undo this come the weekend. Remember: I have many, many wiki documents to manually edit by hand any time this thing is enabled or disabled.

Well, I might as well try to fix the problem before that. It needs a modification in the SyntaxHighlighter scripts.

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: kkfos.aspekt.fi


Top
 Profile  
 
PostPosted: Fri Apr 11, 2014 7:32 am 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 2962
Location: Tampere, Finland
thefox wrote:
koitsu wrote:
I will try to make the time to undo this come the weekend. Remember: I have many, many wiki documents to manually edit by hand any time this thing is enabled or disabled.

Well, I might as well try to fix the problem before that. It needs a modification in the SyntaxHighlighter scripts.

The attached script should fix the issue. Replace the <script> include of shCore.js with an include of this shCore-nesdev.js. I also updated the brush slightly.

Test case: http://wiki.nesdev.com/w/index.php/Scan ... rge_Tables


Attachments:
shCore-nesdev.zip [21.3 KiB]
Downloaded 101 times

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: kkfos.aspekt.fi
Top
 Profile  
 
PostPosted: Fri Apr 11, 2014 4:14 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
Thank you! I'll take care of this later tonight after work.


Top
 Profile  
 
PostPosted: Sat Apr 12, 2014 1:38 am 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
Okay, this is all extremely complex, with many things that could go wrong any moments, so I have opted not to do this yet. Let me discuss the situation.

1. ALL CHANGES THAT DIFFERENTIATE FROM STOCK SOFTWARE MUST BE DOCUMENTED/PATCHES/DIFFS MADE. I do this any time I make a change; this responsibility falls on me, but minimising the number of diffs from stock software is important. Failure to do this results in basically 100% inability for ANY kind of software upgrades to reliably be done. Rephrased: the more one-offs we have to deal with, the more painful upgrades (of MediaWiki, MW extensions, forum software, whatever) become. I think Tepples knows bits/pieces of what the situation is, and I know other bits/pieces.

At this time we already have one diff for SyntaxHighlighter, and I will explain what it contains (two things):

i) Modifies SyntaxHighlighter.php to include "6502" and "asm" SyntaxHighlighter engine references, which refer directly to the engine called "Assembly6502".

ii) Installs a new file, scripts/shBrushAssembly6502.js, which is quite small (1.9KBytes). Your .zip file contains a replacement that appears to be almost identical to this (but not entirely), but the filename is called shBrush6502.js so I would need to rename it (not a problem).

2. The default shCore.js file is 16175 bytes. The shCore-nesdev.js file in the .zip you uploaded is 72517 bytes. This is enormous for Javascript, by the way. After a review of it, it looks to me like it needs to be packed or what's otherwise known as "minified". I have no idea how to pack anything related to Javascript (edit: maybe this is helpful), but there's no way we're putting a 72KByte JS file into place. Minified should be significantly smaller.

3. The author of SyntaxHighlighter, on his site last month, stated that he's working on maintaining the thing once more. However no where in github does he provide the built versions (and I don't know if they're packed/minified).

I won't be changing anything to "include shCore-nesdev.js rather than shCore.js" because that just creates more one-offs. Instead I'll just have to name the thing shCore.js, once a packed version can be provided.

Don't worry about the comments at that Javascript packing site about gzip or webserver compression; that's stuff that has to happen within Apache/on the server side and would require WhoaMan to make changes (it helps substantially). What I used on Parodius was below (and greatly relied upon mod_mime_magic to help properly detect things too):

Code:
LoadModule deflate_module libexec/apache22/mod_deflate.so
LoadModule mime_magic_module libexec/apache22/mod_mime_magic.so

<IfModule deflate_module>
  AddOutputFilterByType DEFLATE text/plain
  AddOutputFilterByType DEFLATE text/html
  AddOutputFilterByType DEFLATE text/css
  AddOutputFilterByType DEFLATE text/csv
  AddOutputFilterByType DEFLATE text/xml
  AddOutputFilterByType DEFLATE application/xml
  AddOutputFilterByType DEFLATE application/xhtml+xml
  AddOutputFilterByType DEFLATE application/javascript
  BrowserMatch "^Mozilla/4" gzip-only-text/html
  BrowserMatch "^Mozilla/4\.0[678]" no-gzip
  BrowserMatch "\bMSIE" !no-gzip !gzip-only-text/html
</IfModule>


Top
 Profile  
 
PostPosted: Sat Apr 12, 2014 4:44 am 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 2962
Location: Tampere, Finland
Quote:
2. The default shCore.js file is 16175 bytes. The shCore-nesdev.js file in the .zip you uploaded is 72517 bytes. This is enormous for Javascript, by the way. After a review of it, it looks to me like it needs to be packed or what's otherwise known as "minified". I have no idea how to pack anything related to Javascript (edit: maybe this is helpful), but there's no way we're putting a 72KByte JS file into place. Minified should be significantly smaller.

Meh, I agree that 70 KB is on the large side, but the JavaScript files typically are static, so the browser will download them once and cache them for a good time. More often than not, the browser will also accept them gzipped.

The reason I didn't compress the JavaScript was that the first packer that popped up on Google (http://dean.edwards.name/packer/) broke the script.

Quote:
I won't be changing anything to "include shCore-nesdev.js rather than shCore.js" because that just creates more one-offs. Instead I'll just have to name the thing shCore.js, once a packed version can be provided.

Feel free to do whatever you want with the files.

Anyway, I guess I should tell you how shCore-nesdev.js was (manually) built:
1) Started with src/shCore.js from the distribution ZIP.
2) Took the file scripts/XRegExp.js from the repo (https://github.com/alexgorbatchev/synta ... 83/scripts) and pasted it on top of the shCore.js file. This dependency was missing from the distribution ZIP, but was probably added to the packed shCore.js file in the build process.
3) Applied my modifications to the shCore source. You can find them from shCore-nesdev.js by searching for "thefox". It's only 3 lines.

There are some build scripts in the SyntaxHighlighter repo to automate the packing/etc, but I couldn't bother to install PHP to run them.

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: kkfos.aspekt.fi


Top
 Profile  
 
PostPosted: Sat Apr 12, 2014 5:39 am 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19099
Location: NE Indiana, USA (NTSC)
koitsu wrote:
the more one-offs we have to deal with, the more painful upgrades (of MediaWiki, MW extensions, forum software, whatever) become. I think Tepples knows bits/pieces of what the situation is, and I know other bits/pieces.

I've tried to help with this by dividing LocalSettings.php on the wiki into two parts: the standard stuff that comes with vanilla MediaWiki above the line, and site customizations (extensions, etc.) below. And I've tried to put a date on each substantial change to LocalSettings.php. Is there something else that I should have been doing?


Top
 Profile  
 
PostPosted: Sat Apr 12, 2014 4:58 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
So here's the status:

1. I've put things into place verbatim, and they do seem to work. That said:

2. I had to Shift-Reload to pick up the .js changes. I was a little surprised by this, given that the file sizes differ substantially (old vs. new) as well as file timestamps, so I'm not sure what's going on there. I think MediaWiki does offer some way to force a cache update for clients, but I don't think that'd affect this because its not the MediaWiki page content itself that changed, only the stuff referenced by <script> (the filenames haven't changed, etc.). That should all be server-side/Apache-side as far as HTTP server vs. client headers go.

3. All the packers I tried did in fact break the Javascript as you said. Some got them down to reasonable sizes (17KBytes), but they'd break with syntax errors / missing semicolon and the like. As such, I'm going to put in a request to WhoaMan to enable mod_deflate (either for all of Apache or just within the nesdev VirtualHost contexts) to rectify the situation. It's the best I can do.

The reason the file size matters so much is both an issue of wasted network bandwidth and overall speed (it now takes 6x longer to download that single .js file, plus the time it takes to parse it/run it on the client). 76KBytes of JavaScript probably isn't going to make things like tablets (ex. Kindle devices) and other embedded devices very happy. I'm crossing my fingers though. If we get complaints, I'm pulling the entire syntax highlighting crap out entirely and will manually go through all the pages and remove the tags (this takes me an hour plus to do).

I think it's utterly ridiculous how crappy all these syntax highlighters are. I keep seeing new ones pop up on the web all over the place on blogs and so on, but god only knows what's going on under the hood there.


Top
 Profile  
 
PostPosted: Sat Apr 12, 2014 5:00 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
tepples wrote:
koitsu wrote:
the more one-offs we have to deal with, the more painful upgrades (of MediaWiki, MW extensions, forum software, whatever) become. I think Tepples knows bits/pieces of what the situation is, and I know other bits/pieces.

I've tried to help with this by dividing LocalSettings.php on the wiki into two parts: the standard stuff that comes with vanilla MediaWiki above the line, and site customizations (extensions, etc.) below. And I've tried to put a date on each substantial change to LocalSettings.php. Is there something else that I should have been doing?

No I think that's correct (I follow your same methodology when it comes to LocalSettings.php), but when it comes to extensions I just wanna make sure any hand-modifications to things get backed up as diffs and so on.

I put everything in /www/vhosts/nesdev.com/wiki/mediawiki_extensions/ as you've probably noticed, and fix perms (664 not 644, since group-write (apache) is the only way we can modify each other's stuff. For example your Cite.tar is 644 right now).

For the file permissions stuff, just stick umask 0002 (default is 0022) in your .bashrc on the server and that should take care of new file creation permissions being correct going forward.


Top
 Profile  
 
PostPosted: Sat Apr 12, 2014 5:24 pm 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 2962
Location: Tampere, Finland
koitsu wrote:
3. All the packers I tried did in fact break the Javascript as you said. Some got them down to reasonable sizes (17KBytes), but they'd break with syntax errors / missing semicolon and the like. As such, I'm going to put in a request to WhoaMan to enable mod_deflate (either for all of Apache or just within the nesdev VirtualHost contexts) to rectify the situation. It's the best I can do.

If you want to try, there's a packer written in PHP in the build directory of Syntax Highlighter repo at https://github.com/alexgorbatchev/synta ... ee/3.0.83/. This was probably used to pack it up for the official release, so it may not break it.

Quote:
76KBytes of JavaScript probably isn't going to make things like tablets (ex. Kindle devices) and other embedded devices very happy.

The parsing/running could very well be even slower if the file is packed, depending on what kind of packing is done. Minifying (white space stripping) is probably the best option.

Now I got a little bit curious why the minifying/packing was breaking the script, so I may check out what's wrong at some point.

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: kkfos.aspekt.fi


Top
 Profile  
 
PostPosted: Sat Apr 12, 2014 5:38 pm 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 2962
Location: Tampere, Finland
OK, I think I found the problem. A semicolon was missing on one of the lines (gotta love JavaScript's automatic semicolon insertion). The attached file should fix the problem. It's not compressed or minified, feel free to do that with whatever you want.


Attachments:
shCore-nesdev-2.zip [20.41 KiB]
Downloaded 90 times

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: kkfos.aspekt.fi
Top
 Profile  
 
PostPosted: Wed Sep 10, 2014 7:04 pm 
Offline

Joined: Sun Sep 19, 2004 11:12 pm
Posts: 19099
Location: NE Indiana, USA (NTSC)
Bump again: it just came up in #nesdev.

In #nesdev, thefox wrote:
The frame and NMIs
the javascript syntax highlighting is messed up on this page, you should be able to hack in a fix among the previous hacks by editing shCore.js
"messed up" meaning :+ gets converted to ;:+

I'm not familiar with the syntax highlighting codebase; koitsu, are you still around?


Top
 Profile  
 
PostPosted: Wed Sep 10, 2014 7:07 pm 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 2962
Location: Tampere, Finland
You can search for "thefox" in shCore.js, there are a bunch of replace() lines. If you add "&#160;" -> " " in there, that should take care of the issue. " " should preferably be a non-breaking space, as that's what the HTML entity 160 stands for.

Unfortunately this would be, like the other existing replace() lines, just another hack, because all HTML entities are not covered.

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: kkfos.aspekt.fi


Top
 Profile  
 
PostPosted: Wed Sep 10, 2014 8:36 pm 
Offline
User avatar

Joined: Sun Sep 19, 2004 9:28 pm
Posts: 3192
Location: Mountain View, CA, USA
I'm still around. I said I'd get to this when I can. And I will. I just have to go through all the pages I added to make use of the syntax highlighter and remove the syntax highlighting bits by hand, followed by removal of the syntax highlighter altogether.


Top
 Profile  
 
PostPosted: Wed Sep 10, 2014 9:19 pm 
Offline
User avatar

Joined: Mon Jan 03, 2005 10:36 am
Posts: 2962
Location: Tampere, Finland
Eh, that was not the point. The bug can be fixed by applying the small modification I mentioned in the previous post.

If you still want to remove the syntax highlighting, though, that's fine by me.

_________________
Download STREEMERZ for NES from fauxgame.com! — Some other stuff I've done: kkfos.aspekt.fi


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic This topic is locked, you cannot edit posts or make further replies.  [ 47 posts ]  Go to page Previous  1, 2, 3, 4  Next

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group