Difficulty Assignment Improvement Project

[swietymiki]

This is the beginning of the most serious thread I've ever written. I'll need your help. From anyone who played any Gruntz Custom Levels.

I'd want to bring up Quest difficulty distribution on this forum. From over 600 maps, based on numbers from the indices, there are 125 levels with "Very Easy" and "Easy" rating. On the other hand, there are 61 levels with "Hard", "Very Hard" or "Frustrating" rating. So this leaves us around 70% of all custom levels as "Medium". And while these are just statistics, playing various maps sometimes left me disagreed with the difficulty given to it. Often you don't even have to play the level, you just look at the map and know that something is not right. I am not blaming anyone for this but I want to fix what is possible to be fixed in the difficulty spread.

Why is it so important that I have to write this thread? Maybe I care about the state of this forum a little too much but I'd want people to give their voice regarding the ratings of various levels, in this thread. When I returned to this forum almost a year ago, I was unlucky enough to find this one damn level "Bombz Away", rated only "Hard", so then I thought that what I created was a piece of cake and titled "The Four Sword Gruntz" as "Medium". Fortunately Zu made me understand that it's not right.

I find a couple of reasons that might have caused this situation. Firstly, I recently read that back in the early days of this forum the difficulty only had 3 possible ratings: easy, medium and hard (thread from 2005). After new difficulty levels were introduced (I can only find traces of them from mid-2006), it is understandable that all ratings weren't properly reassigned. Secondly, there were a lot of creators who only dropped one or two levels and didn't have a lot of experience in custom levels, perhaps didn't pay much attention and just treated the rating as one of many fields that needs to be filled. The problem is that if you don't have knowledge of many other Custom Levels, you do not give the difficulty properly in comparison to other, already released maps. Thirdly, it depends on the skill level of the author. After creating a level as a newbie, you might think it's "Medium", so you give it this rating. But what if this "Medium" is most players' "Very Easy" to "Easy"? What is much worse is when you call something "Medium" and in reality it is much harder, as happens from some experienced designers.

What this non-comparative difficulty assignment creates is that people get discouraged from playing custom levels at all.
I finally get to the point: Anyone can participate and it doesn't require special effort. After you finish playing a custom level and you feel like it is rated incorrectly in comparison to other levels, just state your opinion under this thread. Hopefully more people will share their thoughts here.

[swietymiki]

I'll begin with what I look forward to being changed. Hell of a level, "Bombz Away!", I commented on it in its own thread. I'm entirely sure it should belong to the "Very Hard" category rather than where it is now because it is one of the hardest Gruntz quests and there is no higher rating, other than "Frustrating", but that's a completely different story. A different "Hard" rated level I played, InDaHouse, was still difficult but I could figure everything without a walkthrough. Though I played it a long time ago and I might have overrated it a little, maybe it would be better to use something newer as an anchor for a "Hard" but not "Very Hard" level - like "Find Something To Break Stuff!" (that was the edited part). I am not the only one who had problems with "Bombz Away!", there are as many as 10 pages of people asking for hints.

Generally, Gamer's levels have very small difficulty spread, only from "Medium" to "Hard", and I expect more of them to differ in their real difficulty. Quick look at the smallest map of his: size 20x15, what so difficult can it contain to be called "Medium"?

It would be great to hear opinions of less active users, like Rik, rohit, SwordGrunt, BattlezM... I don't want to force anyone to replay all released levels, just say what you already know. I'll be posting here regardless if I have any doubts on a released level's difficulty.

[Lahyme]

Hey buddy!

I really think the navigation system throught the Custom Levels is really hard and not convenient... (the HTML pages). I wonder if someone could organize better them with SQL language and a DATABASE.

Another thing I think is to get all VERY EASY and EASY levels together. I think everyone could solve them without too much concern. (That would decrease the segregation of pages and links).

What you think swietymiki ?

[swietymiki]

Moving over 600 custom levels and all their files to a new place would be a lot of work that no one wants to do himself. It was definitely suggested in the past and it's rather off-topic.

And what you suggest in the second part is completely opposite to what I meant in the post... Custom levels shouldn't be thrown into one bag and there is a visible difference between the "Very Easy" and "Easy" difficulties. Doesn't matter if everyone can beat them, let people decide what they want to play.

[ILoveGruntz]

As the guy who more or less created the "Frustrating" difficulty, I thought I should chip in.

The current rating system is VERY subjective unfortunately. They are rated mostly by what the designer thought was appropriate, adjusted only if there were a lot of complaints.
But people value different types of puzzles and level compositions differently.

Some would bump up a fairly easy level if it has one annoying timing part. Some would rate levels higher if they try to trick the player into doing wrong things, even if it is easy on your second go. Some try to rate the average difficulty, and gloss over a few very tricky puzzles in a large level.

The difficulty rating in general could probably use an overhaul. Maybe we should add an optional breakdown of the reason for the score.
Something like:

Average difficulty: Medium
Hardest puzzles: Very hard
Common difficulty type: Logical/Timing

Overall difficulty: Hard


Anyways. I think a good start would be to set a few benchmarks that everyone can agree upon before rating. Take the questz levels and rate them in sequence from the tutorial to The Final Battle, and give specific reasons for why they have a given rating. This might not give a good base for the highest difficulties, so throw in a few of the more infamous custom levels for good measure.

[swietymiki]

Good idea, maybe we should try adding a small table like this, at least for new levels. But two problems remain: What is put in it needs to be decided by more than one person to be objective, and that person would probably turn out to be the Beta-Tester, who often has input into the level anyway. Also, old levels remain with the limited rating, and no one will spend their time to update hundreds of released maps.

I never had a long thought on how custom levels relate to standard ones in difficulty but observing map images and playing a few you get a rough outline of what certain difficulty holds. The simplest example is "Very Easy", where levels basically don't require thinking and/or are really small. But when a level is clearly mis-rated I just have to point that out and I hold my opinion on "Bombz Away!".

[SwordGrunt]

On my first days as a Gruntz Designer over 10 years ago, I quickly wiped up many different levels, a large portion of them never being finished. My first level (BeatTheDisgruntled) was almost entirely luck reliant as you had to win 1vs1 battlez with no room to move around, and the next (LavaRiver) had exceedingly difficult timing on toggling bridgez and the like. And with the criticism I received for that and my own experience of playing different levelz, I quickly realized I wanted to go for something else. So I started trying to go for the brains more than for the fingers and dice-rolling, and many times intentionally made the solution to puzzles appear to be based on the latter two, but more attentive solvers could find a way to outsmart that and guarantee a win. To me, the more difficult to figure out and easier to overlook the solution was, the better. But there are a lot of different ways a level can provide difficulty despite how limited the game is to edit.

Timing puzzles are a core section of Gruntz and are boring when too easy, but frustrating when too hard. In my opinion, every level needs to have at least one section where the player has to pay attention and not slack off on their Gruntz' movement or they will lose and have to reload the game, be it a toggling bridge, a rolling ball or being chased by a Toyer. I've tried over the years to find the right balance for this and usually ended up on the easy side for traditional timed bridgez or pyramidz (secret or silver) because I think they should keep you attentive more than actually force you to try multiple times, but I've played plenty of great levelz where timing is your biggest enemy. Standard Levelz approach this with steady difficulty increase but even on the last ones there were no absurd timing puzzles I can remember.

Battlez are another key to the game and occur a huge portion of Questz levelz. In the Standard Levelz, they are frequent and easy (anything you can't beat straight up will have a toy to help you) but I've come to use them with higher difficulty very often in most of my levelz. They are for the most part a strategy game but there can be timing involved as well, such as when you have to hit an Object Guard at just the right time to not get hit back, or quickly run away before getting hit back by a slower opponent until you can recharge. It's an aspect of the game that can combine all of brain, fingers and luck (usually you want to stick to the first two though, I had to playtest BeatTheDisgruntled so many times I quickly started cheating through).

Strategy is what makes Gruntz what it is. I've tried my best to use it as the main factor of every level I make. As a Designer, you need to be creative and come up with a puzzle that is more than just picking up Gauntletz and breaking the rockz blocking your path, or grabbing a toy to give it to the Clubz enemy that might harm you on the way. Sure, those are some things you can do, but what if you give that toy another use later on and have the solver just run past the Clubz enemy, or kill him using Gauntletz? It's not the obvious solution, but it's what the solver will have to figure out by planning ahead and using their knowledge of the game. Of course, you should be careful not to overdo it, as some straightforward areas will let the solver relax and feel comfortable to look at what's coming ahead while not being under pressure. But this is, in my opinion, what really puts the fun both in creating and playing Questz.

Splitting the difficulty rating in three basic areas could go a long way already, but it's not foolproof because this game is subjective by nature. I could find it a piece of cake to zig-zag through crumbling bridgez with Springz, but I'm sure there's someone who doesn't. I have trouble managing too many Gruntz in different areas and there are definitely people who can keep track of them better. That's why it's hard to rate custom levelz and why there are so many "Medium" ones - I don't think that's wrong at all, but I agree there could be several mis-judged levels out there which could use opinions, and anyone is encouraged to voice theirs.

[swietymiki]

Today I took the time to play a bunch of maps that I suspected of having a wrong difficulty rating, mostly small-sized or made by an unexperienced designer. My to-do list is still long but I can already draw some conclusions.
Some ratings were (IMO) surprisingly accurate, like Crazzzy Run from 2001, where a rather small level contains as many as 5 secretz and a lot of timing-based puzzles, so Medium seems fair. However, when there's a short level that basically doesn't require thinking and everything in it is straightforward (even the secrets) then it undoubtedly deserves to go at least one level down. I expect a lot of maps from fairly unknown creators to be overrated because creating difficult puzzles is not something that everyone can do from the start.

My suggestions (feel free to give your opinion on these changes):

X Markz The Spot 8 by GooRoo (Medium -> Easy), opinion given as a comment on its thread
TimebombCRAZY by LittleGruntz (Medium -> Easy), coin collecting simulator, kind of a tryout level, overall nothing difficult but you may get bored with collecting free secrets
Move Off The Island by Gamer (Medium -> Very Easy), this is a really short map with only one actual puzzle, the rating it has gotten is ridiculous
Tiny Lake by Zazor5000 (Medium -> Easy), pretty small and straightforward, only getting through the rolling balls might be a problem
Bombz Away! by Gamer (Hard -> Very Hard), hell of a level - huge, brain-draining and unfair for anyone playing without walkthrough, if that's not one of the most difficult levels then I don't know what is

[BattlezM]

I agree with the notion that there are different categories that represent difficulty in a level. There may be level with very hard timing sections, but very easy battlez/strategy parts. What difficulty does this level get? Is it as hard as the hardest component (the timing) or just an average (so this would be just hard since the other stuff is easy/very easy in comparison)

Personally though when I glance at the difficulty rating of a level, I use it as a gauge how long it might take to complete the level itself. I wouldn't expect an easy level to be anything over 30 minutes, whereas frustrating could be the better part of a day. This is all subjective of course.

However, I'd say that something that isn't as subject would be a difficulty system that ranks on expected experience for an 'easy' time. So I'd put something like iLoveGruntz's levels on 'super veteran', other normal level on somewhere between 'beginner' and 'veteran'. Basically veteran difficulty levels are levels designed with the expectation that you *really* know what you're doing, strategy stuff like 'friendly fire' for instance, would instantly make a level veteran difficulty status.

It's very different than what we currently use, but I think stuff like 'easy-medium-hard' are just always going to be subjective. I'd want the category system, but it almost feels like information overload. The 'beginner-normal-veteran' system is similar to what we have, but I think it describes a little more rigidly the intended difficulty of a level from a designer's view.

[GooRoo]

One thing to remember: there are close to 700 Custom Levelz (including both Questz and Battlez), and each CL has an entry in alphabetical order, graphics set, Release Date, Designer ... and Questz also have an entry in Difficulty, with Battlez in a separate index. So we are talking about over 3,000 updates required (allowing for a rating even for Battlez) to implement a change in difficulty ratings such as have been discussed here. I have recently spent at least one hundred hours in updates to indices ... just to correct errors in a few of the Questz. Involving every single CL in a project would require months for me to implement. Frankly, I have enough things to do, and I am already (Biblicly) on borrowed time (the extra 10 years beyond the "three score and ten"), so have to consider the important, and ignoring much of the urgent.

Rather than a global re-rating, I would prefer individual postings (perhaps by a team of evaluators) in the various Announcement threads with what the evaluators believe is a reasonable idea of how much enjoyment the beginner/normal/expert Gruntzer could expect from playing the puzzle.

The multi-player people have produced valuable insights as to the play value of several Battlez in the quarter year just nearing completion. The same effort, without the updates to indices, would be helpful to people just being introduced to Gruntz. And for those with about the same number of years sucking air (such as I), a note that there is difficult timing issues ... for those with less than nimble fingers would save some frustration for someone now using the "hunt and peck" method of typing. Similarly, a note that "this puzzle contains challenges suitable for a Mensan" might be an enticement for someone capable of matching wits with (the departed) Isaac Asimov.

[swietymiki]

Because of the high number of Questz that we already have, I ruled out a complete difficulty re-rating process. The best thing we can do is give our opinions on played maps and be comparative to other levels (once you solve enough puzzles it becomes easier to rate things), or only apply some changes to the newest creations (which make the most activity anyway).

I already have a couple more suggestions for individual level ratings as a continuation to what I posted on 10th of August. I discovered something rather unexpected - the biggest amount of "Medium" ratings actually doesn't come from unknown designers but from the most popular ones - Gaby, GooRoo, OldOne, etc. made hundreds of levels alone and their difficulty ratings leave a lot to be desired (unless you're gonna tell me that a person can make 300 puzzles on identical difficulty level!), though I'll probably never be able to complete all Questz from any of these creators to be sure.

[swietymiki]

I've been playing more old(er) levels over the last months and decided to list those with unsatisfactory difficulty ratings once again:

The Great Space Walk by Chrystien (Medium -> Easy), probably the only reason for this rating was a short UFO section that takes a few tries to pass or an invisible path for the coins, however nothing else is difficult here
Quit, Or Double by Mikker (Easy -> Very Easy), a very specific level, you can complete it freely with no coins, and for the "Perfect" you mostly need to restart till an enemy gets pushed into the correct tile by your wingz
Lost Vegaz by Chrystien (Medium -> Easy), if we ignore the fact it has no Warpletter (nor coins) at the moment, this level is extremely short and the timing puzzles are certainly easier than in the original Secret Level
Eggz For Two by GooRoo (Medium -> Easy), as I mentioned back in August, there are several unplanned solutions to this map, being given more stuff than necessary makes it much easier for everyone
Fun For Your Feet by Gamer (Medium -> Easy), if the opening puzzle was changed so that you could get both tools every time, there isn't much you need to do to get the "Perfect" if you are able to collect the Warpletter
Bridge Over Troubled Waterz by GooRoo (Medium -> Easy), you only walk around avoiding enemy fire and get rid of other Gruntz to make a passage, I didn't encounter any problems other than the AI's stubbornness
Spicy Spikez by Mikker (Medium -> Easy+), all depends on your pathing ability with springz, for me it was very straightforward, except the northeastern part perhaps
Gaspik Space by Gaspi2 (Medium -> Medium+), opinion posted on the level's thread

[athrun]

So, I intend to actually play some of the Quest levels as I have time and inclination once I am back from my trip. The level collaboration has given me some insight into some of the strategies used in the Quests so I may not be a total dunce trying to do them. I'll likely give my opinion on any I do as well, being more or less a puzzle-dummy who hasn't touched the game Quests (completed without warp secrets) in ages.

Brief opinion without looking at many is that perhaps some of the levels could include a little blurb (without giving anything of the actual puzzle away) about what difficulty it includes? Be it tight timing, a lot of dexterity challenges, or heavy pondering puzzles?

[swietymiki]

This is just a recap of my suggested difficulty changes to levels that I've been posting about over the last 2 months:

Quadrupletz Timez Three by Marie-Claude Bernier (Medium -> Easy)
Small Problem For The Gruntz by Marie-Claude Bernier (Medium -> Easy+)
Casino Road by Zazor5000 (Medium -> Medium++)
Gruntzy Gremez by GooRoo (Medium -> Easy+)
Switchez And Pyramidz by GooRoo (Easy -> Easy+)
Rrrrockz Are Fun by GooRoo (Easy -> Easy+)
Jericho's Wallz by GooRoo (Medium -> Easy+)
Springtime In Cloud City by GooRoo (Medium -> Easy+)
Water Hazard by GooRoo (Easy -> Medium)
Broken Arrowz by GooRoo (Medium -> Easy+)
Ali Baba by GooRoo (Medium -> Medium+)
The Breakfast Club by GooRoo (Medium -> Easy+)
Gentlemen Only, Ladiez Forbidden by GooRoo (Medium -> Medium+)
Cratered Planetz by GooRoo (Easy -> Easy+)
Cupcakez Galore! by GooRoo (Medium -> Easy+)
Uphill Battle by GooRoo (Medium -> Hard)

[GooRoo]

The recent (much appreciated!) spate of video walk-throughs by grosserfluss have brought me back to activity in this Gruntz Forum.

You may see some of the results in the Custom Levelz (Questz) threads. Also I scanned the "other sites" and found some things of interest (to me).

And in this thread I read about opinions about re-rating of some Questz. Which got me thinking about perhaps setting up some rules (guidelines) for ratings.
Feel free to comment on your revisions to my criteria.

Minimum requirements to be rated "Medium":

1. One
2. One acquiring only the (single) ... with some indication of where the trigger spot is, as
3. Crumbling Bridgez to access something of importance ... point and click.
4. Silver Pyramid puzzle ... all fall down (long enough)
5. No more than one per sub-puzzle (area accessed by meeting a requirement) ... more is unnecessarily time-consuming.
6. "Magic Shovel" location must be plainly different from any ordinary . (Otherwise the rating may be marked "Frustrating".)

Earn a "Hard"(er) rating with some of these:

1. Crumbling Bridgez to access something of importance ... with some toggling bridge
2. Adding Objects such as ObjectDroppers, UFOs, etc. for the solver to work around.
3. Silver Pyramid puzzle ... lowered in sequence with direction changes
4. Move through a Pyramid maze with externally controlled lowering and (re-) raising.
5. You can't get there without (unintentional? ) help
6. One Grunt runs around pressing Yellow (Toggle/Hold) Switches while another Grunt helplessly moves through a maze.
7. Uncovering a in a limited space ... uncover the wrong one and you are toast!
8. Send an enemy Grunt somewhere that you cannot (or don't want to) reach yourself.

Earn a "Very Hard" rating with many of these:

1. Adding Objects such as ObjectDroppers, UFOs, etc. for the solver to work around ... in tight spaces with enemy Gruntz attacking, as well.
2. Toggling bridge tiles with zig-zagging path and staggered up/down times
3. Silver Pyramid puzzle ... lowered in sequence with direction changes , hesitations (with Pyramidz rising, catching up to you) ... and even reversing course at times.
4. Move through a Pyramid maze with externally controlled lowering and (re-) raising and enemy Gruntz with ranged weapons harassing you..
5. You can't get there without (unintentional? ) help
6. One Grunt runs around pressing Yellow (Toggle/Hold) Switches while another Grunt helplessly moves through a maze ... perhaps while under attack.
7. Uncovering a in a limited space ... uncover the wrong one and you are toast!
8. Send an enemy Grunt somewhere that you cannot (or don't want to) reach yourself.
9. Have very little time to accomplish all of these things (perhaps with a clock running down!) Examples: Guenther and Paul

Adding to the thread as I think of things.

[swietymiki]

I think what you suggested are good guidelines for designers who are starting out. Sticking to these suggestions should help avoid some mistakes that make levels either too boring or too annoying to complete.

However, I do not see the connection of these criteria to difficulty ratings. Lack of secrets doesn't prevent a level from having super difficult puzzles that take hours to solve, and putting two coins to collect per area wouldn't make me think that the level was any harder or easier.

The guidelines for specific difficulty ratings should be related to level traits that players pay attention to when thinking of custom levels as difficult, because that's what they're for, to help people find the suitable level for them. To elaborate:

• A hard or very hard level would be a level that requires a lot of thinking, with the solver often having to stop and analyze the situation on the map carefully to find the solution. This is to some degree subjective, as some players are better than others at finding the solutions to puzzles, but certain levels have a long history of tricking even very experienced players. Another thing that could make the player think of a level as hard would be having very tight and/or long dexterity sections without a safe place to rest and save your game. These are the places where players have to reload their game a lot and it adds to the overall solve time of the level.
• An easy or very easy level would be the opposite, a level that does not take much thought as it is mostly about walking around the map while doing basic tasks, such as breaking rocks, killing poorly armed enemies and only being met with easily passable obstacles. Someone would play a level like that if they wanted the satisfaction of beating it without having to go through the hardship of solving puzzles.
• A medium level would be somewhere in the middle, not hard enough to make you think that the designer is out to get you, but also not easy enough to make you yawn. It should have some kind challenge in it; if there is no challenge whatsoever, that's what the easy category is for. What exactly makes a challenge is difficult to define, one piece of advice that I once saw in a discussion under a custom level was that there must be a possibility to solve a puzzle wrong and have to redo it.

[GooRoo]

My "Medium" difficulty 'rules' are just a starting point. Your points about 'Hard', 'Easy' and 'Very Easy' I would make as "all of the above (or below)" points in additional table entries. So "too many Coins may mark a level as Easy (annoying)", or (adding to 'Medium' something about Crumbling/Toggling Bridges that is of medium difficulty) "point to the destination and click' (very easy); few togglers (easy); drowning frequently occurs (hard), etc.

"Medium" is my starting (central) point: add your own table entries as you see the other difficulty ratings should have (or be missing).

[GooRoo]

Jul 31, 2019 4:48:58 GMT -8 swietymiki said:

This is just a recap of my suggested difficulty changes to levels that I've been posting about over the last 2 months:

Quadrupletz Timez Three by @marie-Claude Bernier (Medium -> Easy)

I put this one together from 4 'Very Easy' submissions found elsewhere. "Easy" is probably appropriate.

Small Problem For The Gruntz by @marie-Claude Bernier (Medium -> Easy+)

I put this one together from 3 'Very Easy' submissions found elsewhere. "Easy" is probably appropriate.

Casino Road by @zazor5000 (Medium -> Medium++)

What earns the '++' part about this one?

Gruntzy Gremez by @gooroo (Medium -> Easy+)

What is 'ho-hum' which makes the 'Easy+' part about this one? (How would it be improved?)

Switchez And Pyramidz by @gooroo (Easy -> Easy+)

What earns the '+' part about this one?

Rrrrockz Are Fun by @gooroo (Easy -> Easy+)

What earns the '+' part about this one?

Jericho's Wallz by @gooroo (Medium -> Easy+)

What is 'ho-hum' which makes the 'Easy+' part about this one? (How would it be improved?)

Springtime In Cloud Cityby @gooroo (Medium -> Easy+)

What is 'ho-hum' which makes the 'Easy+' part about this one? (How would it be improved?)

Water Hazard by @gooroo (Easy -> Medium)

This surprises me! What earns the promotion about this one?

Broken Arrowz by @gooroo (Medium -> Easy+)

What is 'ho-hum' which makes the 'Easy+' part about this one? (How would it be improved?)

Ali Baba by @gooroo (Medium -> Medium+)

What earns the '+' part about this one?

The Breakfast Club by @gooroo (Medium -> Easy+)

What is 'ho-hum' which makes the 'Easy+' part about this one? (How would it be improved?)

Gentlemen Only, Ladiez Forbidden by @gooroo (Medium -> Medium+)

What earns the '+' part about this one?

Cratered Planetz by @gooroo (Easy -> Easy+)

What earns the '+' part about this one?

Cupcakez Galore! by @gooroo (Medium -> Easy+)

What is 'ho-hum' which makes the 'Easy+' part about this one? (How would it be improved?)

Uphill Battle by @gooroo (Medium -> Hard)

What is the 'hard' part about this one?

[swietymiki]

To answer your questions in detail, I'm afraid I'd have to replay all of those levels again, since I gave these ratings over 4 years ago. But, back in the day I used to post lengthy descriptions from my experiences playing Custom Levels, so perhaps some of your questions could be answered by those. For instance, here's a link to my post regarding Uphill Battle: gooroosgruntz.proboards.com/post/25937/thread

[grosserfluss]

I can start posting descriptions on what I think about the levels I played, that way we can have more opinions on what the difficulty of a level should be. I am not very experienced puzzle solving wise but levels that are on the same difficulty rating vary by a lot in terms of it's actual difficulty and that is noticeable.