Should the old FMO's have been downgraded to VC's?

[MrRubix]

Difficulty can be defined as a function of certain variables, and an accurate weighting of these variables should naturally account for WHY we find certain songs hard, and should also correctly account for expected number of AAA's given how many plays are done by a certain number of players of certain skill levels.

We should be asking "what makes a song easy/hard" and then trying to relatively quantify it.

[Niala]

Originally Posted by MrRubix

Honestly, I'm in favor of a 20-point scale. I feel like 20 would be a good number to account for the variance of different patterns/song lengths/speeds/etc while still being clearly-definable.

The only REAL problem I see with this is that it's not going to indefinitely solve the problem. While what you say is true, eventually, there will be enough people getting to a really good skill level, and will say "Hey, this song's not actually a mid-19, it's a low-19, and this song isn't a low-14, it's a high-13." It just seems like this would, more or less, just prolong the issue.

ALSO! I would like to point out that people don't seem to much care about any difficulties OTHER than FMO's. It makes it seem like people who are above a certain marker (Not everybody, just some) are trying to keep it to a select few, and make that marker higher for people who have trouble reaching it. The FMO difficulty gets constantly debated on, while the other difficulties are just left be, which doesn't make sense, as they're equally as important to the overall schematics of our system as FMOs are.

[MrRubix]

It doesn't even have to be 20 -- any number we choose is inherently arbitrary. But I just feel like we need to set 1 = easiest song we can think of and 20 = maybe something we can't hope to accomplish yet. If we wanted to make it something like 100, I feel like the variance in defining each level would be too great. 20 seems like a number high enough to account for the different levels of skill, yet low enough to not fall victim to the variance issue. This number may need changing. I'm just trying to explain how we SHOULD be judging difficulty.

EDIT: I am saying each level would need to be properly DEFINED, not just arbitrarily associated.

For example, we should try out a song-length weighting, a pattern-type weighting, a song background-weighting, a note-color weighting, a BPM weighting, etc -- which should accurately predict the average performance. The various levels of performance would be defined by these difficulty scales (I am picking 20 just because I predict that going too high would result in overlapping variances). By this I mean a "98" is really no different from a "99". How do we define such a small change? It's almost meaningless. But something like 20 is psychologically and mathematically/statistically more satisfying.

[awein999]

Originally Posted by Niala

The only REAL problem I see with this is that it's not going to indefinitely solve the problem. While what you say is true, eventually, there will be enough people getting to a really good skill level, and will say "Hey, this song's not actually a mid-19, it's a low-19, and this song isn't a low-14, it's a high-13." It just seems like this would, more or less, just prolong the issue.

it would expand the range of songs so that it is less of an issue today. And when it becomes a big issue again like it is now, you can make the scale higher, maybe 30. Or better yet we could just go with the actual song difficulties thread already

if this wasn't an option I am still in favor of those select former fmo's being downgraded

[kjwkjw]

Originally Posted by MrRubix

EDIT: I am saying each level would need to be properly DEFINED, not just arbitrarily associated.

For example, we should try out a song-length weighting, a pattern-type weighting, a song background-weighting, a note-color weighting, a BPM weighting, etc -- which should accurately predict the average performance. The various levels of performance would be defined by these difficulty scales (I am picking 20 just because I predict that going too high would result in overlapping variances).

So the overall improving trend of FFRers won't really be factored into this system?

/confused like an idiot >.<

[Niala]

I like your rationale, Rubix, but that raises the question of how do you properly define the difficulties? I assume by quantifying it in some way, but what way would that be? Taking in different aspects of it, and judging them based on a certain percentile of what the maximum acceptance value and minimum acceptance value of a file could be? Just an idea.

EDIT: Also, you seem to type insanely fast, which is weird as I type 100+WPM, and answer my questions before I ask them every time....

[MrRubix]

Originally Posted by kjwkjw

So the overall improving trend of FFRers won't really be factored into this system?

/confused like an idiot >.<

Hard to say -- the fundamental issue here is that we can't always say something is "hard" if everyone is always finding it "easy." However, it's misleading to a new player to see a song noted as "easy" only to find that it's actually hard to them. Difficulties need to be accurately defined as a function of different variables. If we're good enough, we can AAA a 4 just as easily as we can AAA a 6, so the data there does not really lend us much information about the actual variance in difficulty. I am just suspecting that market forces here play a big role -- I predict we have more Diff 1 AAA's than Diff 2's than Diff 3's, etc.

But what we CAN say is, for instance, a jack of length 4 may be harder than a jack of length 3. A longer song is harder than an easier song. A stream of this type is harder than a stream of this type. This speed is harder than this speed. From there we can derive relative difficulty gauges that should be a bit more accurate.

[sayuncle990]

I honestly don't think it should be factored in at all. I mean, only a relatively small percentage of FFR players are getting that skill boost that is so prevalent in the forums. I don't see myself as a very good player, right? But as I see all of these songs I once thought hard being downgraded, its like the really good people just are getting jealous that more people are getting better. And they want to be seen as elite again. So they try to make the scales harder and more biased towards them.

Does that make sense? It made sense in my head...

[MrRubix]

Originally Posted by sayuncle990

I honestly don't think it should be factored in at all. I mean, only a relatively small percentage of FFR players are getting that skill boost that is so prevalent in the forums. I don't see myself as a very good player, right? But as I see all of these songs I once thought hard being downgraded, its like the really good people just are getting jealous that more people are getting better. And they want to be seen as elite again. So they try to make the scales harder and more biased towards them.

Does that make sense? It made sense in my head...

Again, market forces. We need to look at the trends of performance across all the songs and compare them statistically, and then break down what it is that contributes to a given average performance.

[Niala]

That's basically what I was just saying to him via chatango. If we can systematically give everything a numerical value, and use that as our basis for the difficulties of different files, it should work fine. The only thing is, we would have to come up with a surefire way of judging the quantification of different aspects of files, and making sure that every possible aspect of a file would be taken into consideration, which is a daunting and seems to be an incredibly time-consuming task, realistically.

[MrRubix]

Originally Posted by Niala

That's basically what I was just saying to him via chatango. If we can systematically give everything a numerical value, and use that as our basis for the difficulties of different files, it should work fine. The only thing is, we would have to come up with a surefire way of judging the quantification of different aspects of files, and making sure that every possible aspect of a file would be taken into consideration, which is a daunting and seems to be an incredibly time-consuming task, realistically.

I'd break it down into something like this:

A hard file is:
-Something long and tires you out mentally or physically
-Hard to read
-Difficult to physically execute

Each one of these points can be broken down and analyzed. The hard part would be accounting for different playstyles -- or more generally, things that can't be derived from the stepchart itself. A song like FOTBB is hell when playing spread, but is balls-easy when playing index. Some songs are easy if you switch between styles (like I do sometimes for streamy/jacky files).

Different playstyles effectively throw average difficulty ratings out the window unless you want to define a difficulty as an average derived from market forces which take ALL playstyles into account, naturally. Otherwise, set up different difficulty scales depending on playstyle. Either way is correct, but it all depends on how you want to define difficulty.

[bmah]

Originally Posted by MrRubix

It doesn't even have to be 20 -- any number we choose is inherently arbitrary. But I just feel like we need to set 1 = easiest song we can think of and 20 = maybe something we can't hope to accomplish yet. If we wanted to make it something like 100, I feel like the variance in defining each level would be too great. 20 seems like a number high enough to account for the different levels of skill, yet low enough to not fall victim to the variance issue. This number may need changing. I'm just trying to explain how we SHOULD be judging difficulty.

EDIT: I am saying each level would need to be properly DEFINED, not just arbitrarily associated.

Yeah, I do agree with the general jist of your idea. I had this idea before, but the problem was that there are just too many variables to bother with.
If you want to really quantify the whole thing, let's start by listing variables that take into account the difficulty factor. This will be similar to DDR's Groove Radar. Here's my sample layout:

- nature of the song:

A) overall song length
B) speed of song


- complexity of song:

A) overall simfile arrow density

i) number of streams
ii) stream length
iii) number of jumps within streams (jumpstreams)
iv) single arrow to jumps ratio

B) variability of patterns present

C) number of patterns present / repetitiveness

D) types of patterns present*

i) single arrow-related patterns
-> rolls
-> staircases
-> trills
-> jacks
-> triples
-> gallops
-> crossovers
-> runningman
-> spins

ii) jump-related patterns:
-> repetitive jumps (e.g. as in Scyth of 13)
-> chains
-> hands
-> quads

E) arrow types present (secondary indicator; not really a very accurate factor)**
i) 4ths
ii) 8ths
iii) 12ths
iv) 16ths
v) 24ths
vi) 32nds
vii) 48ths
viii) 64ths
ix) 128ths

*Taking a "big picture" look at the whole simfile is easier than quantifying each specific subvariable, especially considering the fact that many patterns can meld together; i.e. it's hard to quantify a medley of patterns
**This is basically attempting to break down the "straightforwardness" of the song into actual variables. More "straightforward" songs tend to be comprised mostly of 4ths and 8ths, while the more "awkward" songs (e.g. jazz songs) tend to have more 16ths and exotic arrow types.


Again, this is just a sample layout. If people really want to pinpoint a song difficulty, no better way to do it than quantitative analysis. But as I said, it's probably tedious. If someone's up to the job, then that would be cool too.


edit: MrRubix mentioned playing styles, which definitely would change one's viewpoints on what patterns are considered more difficult than others. Still, the best thing to do is to probably make a generality to how easy it is to execute a certain pattern. e.g. runningmen by consensus is generally harder to execute than rolls. You can only be objective up to a certain point.

p.s. for more info on patterns, refer to the FFR Picture Dictionary.

[MrRubix]

The problem is that there isn't enough available data to really make accurate estimates -- if we're going by market forces, we'd need the entire FFR database in order to get data for every play ever performed on every song (since the latest FFR build) in order to get an idea for how people are actually doing on a file.

A more practical solution would be to just set up a system that is "well-variegated and reasonable." E.g. "What is the average accuracy-breakdown for people who play this song?" accounting for selectivity bias (e.g. maybe most people who play this file are of a certain skill level).

The main problem is that there is often so much debate over "This file should be 9! No, 10!" when it might be easier if we define what exactly those numbers entail. It's a classic problem of using numbers as ratings -- they have no inherent meaning unless we give the numbers meaning, and I think we can give a more accurate meaning if we define what difficulty is.

[bmah]

Yeah, as I said in my edited post though, you can only be objective so far. No one method can really satisfy everyone. There WILL be debate. It's best to specify some parameters, and then make generalities on some of the pickier factors.

edit: I made more edits to my first post; read that first guys
@Niala: see what I have to say about multiple playing styles in my first post

[Niala]

I like your ideas bmah, but it seems to me that they would fit pretty much inside points 2 and 3 of Rubix's. You're right, it would be long, tedious, and incredibly difficult to do that for every chart. Plus, there's the issue Rubix stated, different play styles, however, I think it's pretty safe to say that the high majority of players are spread players, and we can judge based on that, now.

[OneHandNow]

example: Bus Rides with People

[MrRubix]

Again, there will be debate, yes, but there will be less debate if we tie the difficulty ratings to the score performances on the songs, which is what I mean by "market forces." The best way to see how hard a song is is clearly how well people do on it!

[awein999]

Originally Posted by OneHandNow

example: Bus Rides with People

many people think that should be fmo though and it's a newer file.

yeah I agree rubix. That's why I think most of those fmo's were rightly downgraded under the current system.

and that's why I think szerencsetlen should be fmo lol

[bmah]

Originally Posted by 0

since there's already a 1-99 difficulty system, what if they just used those difficulty factors to find what the difficulties of the songs should be on the 1-99 scale, and then use those numbers to sort the songs into the more broad difficulties (68~75 = VC, 76~83 = FMO, etc)

You've just reminded me, my first post said nothing about difficulties. Well, I assume that someone would find a way to add up all of the factors and determine the overall difficulty. For instance, song complexity takes the cake - definitely the biggest factor. Song length is a lesser factor. Eventually, we could come up with an actual difficulty - one that we know we've tried our best to be objective as possible. Further debate on the difficulty would not have to be highly considered, because we assume that such arguments are just products of personal preferences at that point, and you can't do anything about that.

[bmah]

Originally Posted by MrRubix

Again, there will be debate, yes, but there will be less debate if we tie the difficulty ratings to the score performances on the songs, which is what I mean by "market forces." The best way to see how hard a song is is clearly how well people do on it!

This does make sense, but don't forget to take into account the overall number of players of course. This is obvious when you compare a public song vs a token song, or an early-released song vs a new song. And then there's song popularity, which would definitely impact the number of good scores present. An obscure song that not many people play would be a different story, and this might even override the clues provided by the simfiles difficulty (e.g. an unpopular, but easy song may not have as many AAAs as it should. At the same time, there can be a multitude of sucky players, so the ratio of good to bad scores is skewed).