Open up Dan ranks for project-level Kata

[DonaldKellett]

Back in 2014, Codewars administrator @jhoffner envisioned the existence of project-level Kata with dan ranking instead of the usual kyu ranking. And thanks to Subtyping Mutable References which was authored just now, the existence of de-facto project-level Kata has become a reality:

• The solution to this Kata (not even taking into account the preloaded code) took me a full day and more to prepare even by constantly referring to related sources and sometimes copying/modifying code snippets from those sources

• The current solution to this Kata spans a whopping 884 lines of code even after employing advanced automation techniques to try to reduce the line count to a minimum

• The content of this Kata is based on Programming Language Foundations, an e-Book whose content is claimed to be found in typical postgraduate Computer Science courses:

  Programming Language Foundations (volume 2, by a set of authors similar to this book's) covers material that might be found in a graduate course on the theory of programming languages, including Hoare logic, operational semantics, and type systems.

  (source)

• In order to learn the necessary knowledge to author this Kata, I spent about 2 months on end grokking the first 2 volumes of Software Foundations

After careful consideration, it is of my belief that ranking it 1 kyu simply does not do it justice. @kazk Please consider opening up the ranking options of 1-4 dan this time, because if this Kata cannot be ranked at dan level then I don't know if anything else ever will.

[kazk]

if this Kata cannot be ranked at dan level then I don't know if anything else ever will.

I believe what Jake meant was to introduce a different category of challenges with a different scale and use dan for those. By "project level", I believe he had a scaled down version of "real world"ish projects (Rails, Django, Spring, etc.) in mind. On Qualified, we support this kind of challenges and have a decent web IDE that allows you to work with multiple files, but I honestly don't think this will fit Codewars (currently at least) other than purely educational/practice challenges.

Anyways, I don't think he had the size of the solution or the difficulty as a differentiator. So I'm also pretty sure that he wasn't going to treat it as a simple extension to kyu scale in terms of users/kata ranks.

I personally think this will be too confusing so I'd rather have new categories to have own names for their scales or have it explicit ("1 dan in projects").

My opinion is the same:

While I agree that ranks used on Codewars have various issues, I don't think adding few dan level will fix anything. I wish we had more flexible and systematic ways to describe the difficulty. Trying to fit challenges from different areas and different languages into a single scale doesn't work.

I also don't think "could discourage Codewarriors from authoring and/or solving the more difficult ones" will be an issue. I don't think most of us are doing for honor points. We simply like to solve/create interesting problems.
#1769 (comment)

Codewars' gamification is broken like most of the other things. Introducing new ranks won't fix anything. It needs fundamental changes, reorganizing and reranking, but this requires too much work for us (me) to handle at the moment.

More realistic approach might be to come up with something to complement the current system like achievement badges. If "Codewars Battleground" (experimental invite-only competition for Codewars Red users taken on Qualified), is successful, (I think) we're going to introduce some kind of trophy/badge so we might get time to work on it.
Achievement badges are more suitable to handle special cases like this.

Maybe we can create https://github.com/Codewars/badges to maintain the list of badges. Allow admins and hand picked users to assign badges to kata (issues on that repo can be used to suggest/discuss). Then add a section on the profile for badges. We'll have to keep it simple at first, but this might be possible with relatively low effort.

What do you think? Badges are also great to show detailed abilities like Coq: Proved {insert something to be proud of} which I think is way more meaningful than solved 1 dan Coq kata where 1 dan is assessed by few users for a difficulty within Codewars.

[DonaldKellett]

@kazk Special badges for exceptional achievements sounds excellent to me. Do keep me posted on updates regarding "Codewars Battleground" and badges, cheers 👍

---

Side question: Is it still possible to enter the experimental "Codewars Battleground" at this stage? I know it was announced a while ago but I ignored it back then and would like to try it out now if given the chance 😅

[kazk]

Yeah, I'm pretty sure you can and we'll most likely show another popup in the future, but @jhoffner and @andrew-qio knows more about it.
We had a pilot JS competition for intermediate level users so far and I've been only involved as one of the judges.

[Voileexperiments]

After careful consideration, it is of my belief that ranking it 1 kyu simply does not do it justice. Please consider opening up the ranking options of 1-4 dan this time, because if this Kata cannot be ranked at dan level then I don't know if anything else ever will.

Alternatively, can we solve the actual problem and deflate the ranks for all existing Coq katas? Even a relatively simple Coq kata is ranked like 4 kyu and it's way too much (the same can be said about Haskell katas). Basically you all (as in, the first Haskell/Coq users) have self-inflicted this upon yourselves, not to mention that there are basically only <10 Coq users on CW.

Maybe we can create https://github.com/Codewars/badges to maintain the list of badges. Allow admins and hand picked users to assign badges to kata (issues on that repo can be used to suggest/discuss). Then add a section on the profile for badges. We'll have to keep it simple at first, but this might be possible with relatively low effort.

What do you think? Badges are also great to show detailed abilities like Coq: Proved {insert something to be proud of} which I think is way more meaningful than solved 1 dan Coq kata where 1 dan is assessed by few users for a difficulty within Codewars.

There should be more recognition and encouragement to people who introduces new types of katas into the game, author conceptually difficult and original katas that changes the meta around the users. As it is right now the system is still encouraging people shitposting katas and reposting entry-level problems from other sites to CW without attribution. That's really not helping the situation.

In comparison, I don't think it's that needed to give out badges to particular person. After all, someone distinguished enough will get 1 dan or even 2 dan, which is already good enough as a badge.

---

If "Codewars Battleground" (experimental invite-only competition for Codewars Red users taken on Qualified), is successful

I submitted the form for it but still haven't received any responses. What's the plan for that?

[DonaldKellett]

Alternatively, can we solve the actual problem and deflate the ranks for all existing Coq katas? Even a relatively simple Coq kata is ranked like 4 kyu and it's way too much (the same can be said about Haskell katas). Basically you all (as in, the first Haskell/Coq users) have self-inflicted this upon yourselves, not to mention that there are basically only <10 Coq users on CW.

While I do agree that there was overranking of Coq Kata early when it was first introduced to Codewars (e.g. A+A=B+B so A=B? Prove it!, Google Interview and maybe A random fact about filtering, This square is perfect), the rankings have already stabilized since then and I don't see that many newly approved Coq Kata ranked over 4 kyu. Considering that many of the approved Coq Kata lately (mostly ranked under 3 kyu) usually takes a seasoned Coq user at least an hour or two to complete and sometimes even requires reading academic papers to understand the situation at hand before cracking on the Kata which takes at least a few hours of concentration and serious thought, I'd say that the majority of recently approved Coq Kata are quite suitably ranked and do not need any more rank "adjustment"s or deflation.

Regarding your statement that there are currently <10 Coq users on CW, simply taking a look at the "Warriors Trained" metric of nontrivial Coq Kata such as Finding the maximal program reveals that there are at least 24 Coq users on CW who considered themselves competent enough to attempt it. Considering that the Kata requires a careful reading of at least most of Logical Foundations or equivalent, I'd say that at least most of those 24 Codewarriors have sufficient experience in Coq to be considered "Coq users" on CW.

In comparison, I don't think it's that needed to give out badges to particular person. After all, someone distinguished enough will get 1 dan or even 2 dan, which is already good enough as a badge.

While introducing additional badges to CW is by no means necessary, I don't see how they are detrimental to the site either, so why not try it and see how it goes?

[Voileexperiments]

Considering that many of the approved Coq Kata lately (mostly ranked under 3 kyu) usually takes a seasoned Coq user at least an hour or two to complete

I'm not a seasoned Coq user by any stretch (everyone in gitter chat knows this) and I still can complete most of them. You're massively over-estimating the skill level.

... and sometimes even requires reading academic papers to understand the situation at hand before cracking on the Kata which takes at least a few hours of concentration and serious thought

Completely false. I just fumble through the Coq katas with brute force and willpower, frankly. I don't even know how this have to do with academic papers, since I haven't even read any of them! Now if you tell me sometimes even requires reading LF/PLF then it'd be more believable, but LF is intended to be beginner material starting from complete zero. I'd say 5-star problems in LF is roughly about 4 kyu.

The point is, Coq and Haskell kata ranking is basically the same as the general kata ranking back in 2013/14: anyone with a slight hand-on experience with basic Haskell/Coq can complete almost all of the blue/purple ones with enough determination and fumbling. That's not how the ranking should work: you're making the purple katas a pushover to actual experienced users, and I'm definitely not one. Novice users can go struggle against the yellow katas.

---

While introducing additional badges to CW is by no means necessary, I don't see how they are detrimental to the site either, so why not try it and see how it goes?

...because I'm the target user of this feature? If you want to know what badges are actually wanted, why don't you ask the users most likely to receive them (aka me) what they want? 🤔 🤔 🤔 🤔 🤔

Also I think you missed the paragraph above, and you should read it again. What I meant is I don't want badges that tells me the I've done many katas; 2 dan is already a badge that indicates it (and, by extension, kata completion leaderboard) so it'd just be redundant and useless. I want badges that signifies other qualities that cannot be reflected through the leaderboards, like the actual quality and originality of the katas written, as well as site contribution.

[DonaldKellett]

I'm not a seasoned Coq user by any stretch

While you might not be the most experienced Coq user on CW, I'd say that you have plenty of experience with Coq considering that you were able to complete the majority of exercises in LF which is by no means a simple feat.

but LF is intended to be beginner material starting from complete zero

LF is indeed aimed at those with little to no prior Coq experience but it is much more comprehensive than the typical online Coq tutorial so anyone who manages to complete it are well-equipped to conduct a range of interesting, non-trivial proofs. If we measured the ranking of a Kata based on how much knowledge one initially possesses then nothing would ever be ranked above 4 kyu since everyone has to start from nothing.

The point is, Coq and Haskell kata ranking is basically the same as the general kata ranking back in 2013/14

Keep in mind that the massive rank re-adjustments and deflation of more mainstream programming languages on Codewars (JS/Python/Ruby, C#/Java and the like) only really began after your arrival to Codewars and was pushed by a few power users apart from you - I'm sure that at least a significant fraction of power users do not (fully) agree with your re-rankings, though of course no one took action to prevent it since you're one of the very few users who would actively re-rank (down-rank) existing approved Kata just to enforce what you think is the ideal ranking standard. I certainly do not think that doing the same thing to Coq/Haskell/Idris/Agda/whatever would be appropriate to the interests of the community.

...because I'm the target user of this feature? If you want to know what badges are actually wanted, why don't you ask the users most likely to receive them (aka me) what they want? 🤔 🤔 🤔 🤔 🤔

You're a target user of this feature and certainly not the only one so get your articles clear. As another potential target user of this feature, I'd say that it is a good idea to try to introduce some additional badges and see how it goes.

Also I think you missed the paragraph above, and you should read it again. What I meant is I don't want badges that tells me the I've done many katas; 2 dan is already a badge that indicates it (and, by extension, kata completion leaderboard) so it'd just be redundant and useless. I want badges that signifies other qualities that cannot be reflected through the leaderboards, like the actual quality and originality of the katas written, as well as site contribution.

Of course, any additional badges introduced to Codewars would have to be meaningful. I think most would agree that badges like "Authored your first Kata" or "Completed 1000 Kata" are completely pointless. That being said, I personally think that earning a badge for solving a particularly difficult or interesting Kata that its ranking cannot fully reflect shouldn't be out of the question.

[Voileexperiments]

I'd say that you have plenty of experience with Coq considering that you were able to complete the majority of exercises in LF which is by no means a simple feat.

LF is indeed aimed at those with little to no prior Coq experience but it is much more comprehensive than the typical online Coq tutorial so anyone who manages to complete it are well-equipped to conduct a range of interesting, non-trivial proofs.

????? LF is entry level material. That's like saying any college CS student who has followed through entry level data structure material has plenty of experience of coding; it's absurd and is a massive inflation of the actual skill level.

If we measured the ranking of a Kata based on how much knowledge one initially possesses then nothing would ever be ranked above 4 kyu since everyone has to start from nothing.

That's a strawman. By similar argument every kata should be ranked 8 dan because to people who even struggle with Scratch, all these "coding stuff" you do is arcane magic. Clearly 8 dan material.

Also, if your reason of opening dan level is "because we need higher level tiers to make us feel we're not useless", that's a very bad idea. It's also the entire issue about the kata ranking inflation. Now, you can move the kata upwards in the rank scale instead of downwards, but there are much more katas that have over-estimated ranks than katas with under-estimated ranks (not to mention that the latter are usually old katas that need significant revamp before they have acceptable quality), so you're making me and a few other people who do most of the clean-up stuff doing more work for nothing.

I'm sure that at least a significant fraction of power users do not (fully) agree with your re-rankings

Can you stop putting strawman in your so-called "arguments"? We're not talking about whether a particular kata should be re-ranked, but whether katas are over/under-ranked in general and whether (or how) they should be re-ranked. You're sounding like you're trying to invalidate me instead of the topic itself, which is, frankly, pathetic.

Regardless of whether some katas should be re-ranked or not (which is not my concern), many people have already expressed interest about re-ranking (see #157), so it's not something that I made up.

As another potential target user of this feature, I'd say that it is a good idea to try to introduce some additional badges and see how it goes.

Adding badges take time and effort. You can suggest adding every trivial action as badge but that's counter-productive, pointless, and deflates the value of the badge. Hence I'm saying if we're adding badges, we should add the most meaningful ones, and badges for kata completion is not very meaningful because we already have overall/language ranks. It's even called "profile badge":

I personally think that earning a badge for solving a particularly difficult or interesting Kata that its ranking cannot fully reflect shouldn't be out of the question.

So, are badges supposed to be indication of distinction, or a checkpoint that indications you have completed a set task? If you assign badges to one kata or a fixed set of katas, they become a checkpoint, which is completely different from even the standard ranks we have right now (you don't claim to be 1 kyu by doing a specific set of katas).

I don't see what purpose do you intend the role of the badges to be in your proposal, so I don't think you've thought about it thoroughly.

[DonaldKellett]

Also, if your reason of opening dan level is ...

I'll agree that my proposals to open up dan ranks are somewhat geared towards my own interest instead of the community's 😅 I've actually given up on this idea since @kazk mentioned about the plan to introduce additional honorary badges for outstanding achievements or contributions to the CW community so I'll close this Issue to avoid any further misunderstandings.

I'm saying if we're adding badges, we should add the most meaningful ones ...

I don't see what purpose do you intend the role of the badges to be in your proposal, so I don't think you've thought about it thoroughly.

@kazk Perhaps we could open a discussion thread on the topic of additional badges and discuss about it there?

[jhoffner]

I believe what Jake meant was to introduce a different category of challenges with a different scale and use dan for those. By "project level", I believe he had a scaled down version of "real world"ish projects (Rails, Django, Spring, etc.) in mind. On Qualified, we support this kind of challenges and have a decent web IDE that allows you to work with multiple files, but I honestly don't think this will fit Codewars (currently at least) other than purely educational/practice challenges.

Anyways, I don't think he had the size of the solution or the difficulty as a differentiator. So I'm also pretty sure that he wasn't going to treat it as a simple extension to kyu scale in terms of users/kata ranks.

This is mostly correct. I didn't plan on resetting the scale though, so that 1 Dan was easy project challenges, my original thinking was that we wouldn't be able to approach Dan level challenges without project sized challenges. I'm talking challenges that would take you days, maybe a week, to complete. So the scale would be continuous, and it could be possible that simple project challenges were only kyu ranked.

Codewars' gamification is broken like most of the other things. Introducing new ranks won't fix anything. It needs fundamental changes, reorganizing and reranking, but this requires too much work for us (me) to handle at the moment.

I agree, introducing new ranks now is too much. Especially without a systematic restructuring that is too much to handle right now.

I have some regrets about holding dan level rankings back in the first place. At the time it felt like trying to determine what a 8 point difficulty scale looked like, in a way that a community could manage, was challenging enough, much less a 16 point scale. Originally I tried to lay out criteria for what constituted a rank. So certain subject matters would qualify for a certain ranking. You see that criteria when you select a ranking on a beta kata. In the end what has happened though is that the community has grown far beyond whatever intentions I originally had, and without the availability of dan ranks, the entire scale has been pushed down (deflated). No one seems to pay any attention to the criteria listed. The criteria itself is surely flawed, and needed more collaboration on what it should be. At the time that it was created the community was tiny, and we only had a few power users.

The scale is now to the point that many (most?) white kata are too difficult for true beginners, the original intended audience for 8 kyu. Maybe if dan levels had been made available, or at least a few dan levels made available, the deflation would have never happened. What 4 kyu means today is NOT what I had intended it to mean, in terms of difficulty.

More realistic approach might be to come up with something to complement the current system like achievement badges.

I fully agree, and I'll have to go dust off some notes I had about this. In the beginning my co-founder and I had some disagreement about using badges. He thought they were cheesy, but thats because a lot of gamified sites do them poorly. If done right, these can be explicit, much less open to interpretation, and very rewarding to achieve.

The main issue with kyus is that they are so open to the personal experience of how difficult something is. What is difficult for some is not equally difficult for others, one person may find one challenge harder than another, while another person feels the complete reverse. There is also issues with how people feel about themselves. If they feel like they are only an intermediate developer, and they were able to easily complete something - well then it must be easy. When in reality, they very well may just be too humble or insecure about their own abilities. The Kyu system has an emotional element that can't seem to be avoided, but badges/achievements could likely be designed to avoid that.

There should be more recognition and encouragement to people who introduces new types of katas into the game, author conceptually difficult and original katas that changes the meta around the users.

I agree that this is a major issue. The core intention of Codewars was not to just be another code competition site, but to actually move passed the same old algorithmic challenges that everyone else is doing. In addition to achievements, having a bounty system would also be nice (which leads to earning an achievement). To actually encourage people to fill specific gaps that we are missing.

Maybe we can create https://github.com/Codewars/badges to maintain the list of badges. Allow admins and hand picked users to assign badges to kata (issues on that repo can be used to suggest/discuss). Then add a section on the profile for badges. We'll have to keep it simple at first, but this might be possible with relatively low effort.

Perhaps we could open a discussion thread on the topic of additional badges and discuss about it there?

Yeah doing this in an open/crowd sourced format sounds like a great idea. Maybe we do something like create a JSON format and directory structure for how achievements should work (criteria needed, points earned, rank earned, name, etc). If we can get some designers on board, maybe we can open source the achievement badge design as well (if we do designed badges at all).