It is currently Tue Jun 19, 2018 9:35 am

All times are UTC - 7 hours





Post new topic Reply to topic  [ 14 posts ] 
Author Message
 Post subject: Assessment for Judging
PostPosted: Mon Feb 19, 2018 9:42 pm 
Offline
User avatar

Joined: Sat Mar 30, 2013 12:24 am
Posts: 342
Being a teacher, assessment methods (rubrics, grading scales, and the like) are all of particular interest to me.

How are you each approaching judging?

What factors are you including in the overall categories, etc.?

_________________
www.mteegfx.com


Top
 Profile  
 
PostPosted: Tue Feb 20, 2018 8:17 am 
Offline

Joined: Mon May 27, 2013 9:40 am
Posts: 457
It's hard to tell. I usually begin from top to bottom. I usually pick up muy most favourite in each category and give it full marks if I think it deserves it, or almost full marks. Then I pick up my next favourite, and measure how much "worse" it is so I give it the same, 1 less or 2 less or ...

When I'm done I reconsider my decisions. Maybe I was too harsh with my least favourite. Maybe the entries awarded the less points should have higher marks.

So it is quite relative. I don't know if this is a good or bad method, but it's the method I use most of the time in every task where I need to qualify or simply pick an arbitrary order to stuff.

On top of that, I think each one has a different approach to what topic or category is most important. For example, I don't care if a game is original as long as it is fun. Well, I do care that it is original, but that's less important. I don't care if it's technically top-notch as long as it looks good and plays great, and feels polished and complete.

_________________
http://www.mojontwins.com


Top
 Profile  
 
PostPosted: Tue Feb 20, 2018 8:47 am 
Offline
Formerly WheelInventor

Joined: Thu Apr 14, 2016 2:55 am
Posts: 1606
Location: Gothenburg, Sweden
One way, though it requires a bit of prequisite legwork, is to set up atomized and universally functioning merits, and assign them a value (nominally 1). Each merit which you find true rewards a point (or a half, or whatever value you've assigned to that merit). Since some categories can reward more points than others, you'd either have to come up with more merits or attribute them with more points each.

Ie.
the apple was:
-rinsed (1pt)
-good for baking (1pt)
it had:
-an appetizing colour (1pt)
-a nice scent (1pt)
-a distinct flavour (1pt)
-a nice texture (1pt)

Maybe if the apple had a really nice scent, it might be worth 2pts.

Or the merits could be accumulative, perhaps like this:
-the student has grasped the elementary concepts needed to pass the course and continue with the next (threshold 1pt)
-the student has been able to reflect on the concepts and relate them to other related fields of study (threshold 3pt)
-the student has achieved a confident and profound understanding of the subject. (threshold 5pts)

_________________
http://www.frankengraphics.com - personal NES blog


Top
 Profile  
 
PostPosted: Tue Feb 20, 2018 8:59 am 
Offline
User avatar

Joined: Fri May 08, 2015 7:17 pm
Posts: 2091
Location: DIGDUG
I ordered them by how much I like them, then adjusted their scores until the numbers came out the way I wanted. Unfair? maybe.

I also subtracted points for bugs at submission time. Sorry. I know alot of them had been fixed before voting, but deadlines are deadlines. I'm a stickler.

_________________
nesdoug.com -- blog/tutorial on programming for the NES


Top
 Profile  
 
PostPosted: Tue Feb 20, 2018 10:06 am 
Offline
User avatar

Joined: Sat Jan 09, 2016 9:21 pm
Posts: 399
Location: Central Illinois, USA
dougeff wrote:
I also subtracted points for bugs at submission time. Sorry. I know alot of them had been fixed before voting, but deadlines are deadlines. I'm a stickler.


I think it was mentioned somewhere that we should be rating them based on the version available at the deadline/submission, and not based on later improvements, so I agree with this.

_________________
My games: http://www.bitethechili.com


Top
 Profile  
 
PostPosted: Tue Feb 20, 2018 2:22 pm 
Offline

Joined: Mon May 27, 2013 9:40 am
Posts: 457
That's why I haven't still commited to introduce any of the enhancements / corrections suggested for my entry. I will do when the results are posted, for the cart.

_________________
http://www.mojontwins.com


Top
 Profile  
 
PostPosted: Wed Feb 21, 2018 8:55 am 
Offline

Joined: Fri Sep 30, 2016 8:57 pm
Posts: 82
i am also holding off on updates until voting is completed, and will be voting on submissions, not edited versions.


Top
 Profile  
 
PostPosted: Sat Feb 24, 2018 8:27 am 
Offline
User avatar

Joined: Sat Mar 30, 2013 12:24 am
Posts: 342
I feel bad voting on games I haven't been able to beat.

Is it appropriate to PM requesting savestates for endings or that show essential content a developer might want included in the judging of their game?

Did anyone get stuck in Grunio? If so, I'd gladly PM a password list and a savestate that shows the all-carrots ending.

_________________
www.mteegfx.com


Top
 Profile  
 
PostPosted: Sat Feb 24, 2018 11:49 am 
Offline

Joined: Tue Oct 06, 2015 10:16 am
Posts: 746
I only got about two thirds in on Grunio, at that point I just couldn't figure out any solutions. No need to send a password list though.


Top
 Profile  
 
PostPosted: Sat Feb 24, 2018 1:40 pm 
Offline

Joined: Mon May 27, 2013 9:40 am
Posts: 457
There's a cheat in Miedow you can use to skip levels. It's not that hard to find (specially if you check the sources ;) ), but if somebody is interested just PM me.

The ending sequence (which goes from the moment Cheril "awakes" after level 10 to the end and which is playable) is quite nice to experience.

_________________
http://www.mojontwins.com


Top
 Profile  
 
PostPosted: Sun Feb 25, 2018 1:08 am 
Offline

Joined: Sun Nov 23, 2014 12:16 pm
Posts: 242
I am unfamiliar with voting on homebrews or the contest in general. Can anyone vote? When is the voting? Are there rules explaining the voting and contest process?


Top
 Profile  
 
PostPosted: Sun Feb 25, 2018 1:58 am 
Offline
User avatar

Joined: Sat Mar 30, 2013 12:24 am
Posts: 342
http://nesdevcompo.nintendoage.com/rules/2017/#judging
Although judging is not open for anyone (essential for preventing it from becoming an internet popularity contest, IMO), I'd still love to hear your assessment of our entry at least.

_________________
www.mteegfx.com


Top
 Profile  
 
PostPosted: Sun Feb 25, 2018 1:15 pm 
Offline

Joined: Mon May 27, 2013 9:40 am
Posts: 457
I think it would be nice if people posted their own ratings just for fun, specially if commentary was done. I always like to know what people think about my design choices because, as some of you will agree, working for some time in your project basicly blindfolds you.

_________________
http://www.mojontwins.com


Top
 Profile  
 
PostPosted: Thu Mar 01, 2018 11:03 am 
Offline
User avatar

Joined: Sat Mar 30, 2013 12:24 am
Posts: 342
Thanks for the insight into your approaches.

Judging was definitely enjoyable. It's nice to be motivated to take the time to play and analyze these games more than I may have gotten around to otherwise.

Ɓukasz and I assessed separately and then planned to submit the average of our scores. I was caught a bit off guard by being restricted to whole number scores. At first, I'd just planned to round up any category entry that fell between whole numbers. But then I realized some games were getting total score boosts of 1.5 points or more (as they had multiple categories being rounded up). So, then we decided to round each (non whole number) category up or down to ensure that no game received more than a half point advantage from rounding.

As for my half of our judging, I assessed each game independently of each other, using the attached sheet which broke down most categories into smaller subcategories and a scale for each.
Attachment:
NesDevMTeeScorecard.pdf [23.08 KiB]
Downloaded 60 times

Anyway, I printed these and penciled them in fairly messily during my last play session, but if anyone wants my individual assessment of their entry, I'll gladly type up a legible copy with notes when I get a chance and shoot it your way.

_________________
www.mteegfx.com


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 14 posts ] 

All times are UTC - 7 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group