User Rating: Motivation and Computation

User Rating calculations

This blog explains the rationale behind the user rating, and presents a rough sketch how it works. We assume that you are somewhat familiar with the concept of route rating, which represents the effort needed to hike that route. The route rating is based on several factors such as length of the route, elevation gain, sac scale, via ferrata, path visibility etc.

Motivation:

The user rating was introduced to serve the following two purposes:

  • We wanted to have a rating system for hikers so that they can compare their rating and that of the route to decide which routes they are comfortable with. This does NOT mean that they can overlook the route description (i.e., difficulty level and other requirements) and only rely on ratings. Rather, the rating should be used to filter the set of routes while searching candidate routes for a hike on website.
  • A user can also use the rating to track their progress. After each hike they are awarded some rating points based on their current rating and that of the route, which could serve as a motivation factor.

[Of course, we cannot estimate the rating of those users, who haven’t done many hikes through the website. However, in the long run, most users will have enough hikes on the website for their rating to be accurate]

Rating Scale:

The lowest possible user rating is 700, which is also the default rating of a new user. Since the rating of Eibsee hike is a little less than 700, we can say that any hiker should be comfortable with this hike, and can do it without any preparation.

If a user has 1200 rating, then they should be comfortable with the routes in the rating range 1200 ⨦ 100, i.e., they can do Schafreuter, Gamsjoch, Notkarspitze, Rubihorn with ease, and will find them enjoying (i.e., mixture of challenge and still within reach).

An advanced user with rating around 1800, should be able to do hikes such as Zugspitze, Alpspitze via Matheisenkar, Estergebirge on a regular basis without having any pre or post-hike stress/issues.

There is no upper limit on the rating of a user, however, it gets increasingly difficult to obtain higher rating. There are levels assigned with rating ranges, which are shown in the following image.

User Rating Distribution

How Rating Changes:

Once a user completes a hike, their rating is increased. This does not happen immediately after the hike, but five days after the event. The delay of five days is to give the organizer enough time to remove users who didn’t complete the hike. Also during this time event participants can upload pictures (if an event has no pictures, then the rating is updated only after admin approval).

Let’s say a user, whose rating is 900, does a route with rating 1500. Then, we know that the user is capable of doing much harder hikes than their rating and it should be increased significantly. However, it will not be increased to 1500 but to approximately 1100. The rationale behind this is that the user managed to do a hike of rating 1500, but it is possible that they struggled on this route, or they really had to push their limits. If they are consistently doing hikes of rating 1500, then soon their rating will move to 1500 following a series like below:

900 → 1100 → 1245 → 1348 → 1421 → ... → 1500+

This way the user rating does not get inflated just because of a single hike, but actually converges to a rating of hike, which they are doing consistently, and are fairly comfortable with.

Let's consider another example: Suppose two users with rating 1200 and 1300 do a hike with rating 1600, then their rating will be increased as follows:

1200 → 1346 (+146)

1300 → 1418 (+118)

Note that, the user with 1200 rating got a higher increment (146) compared to the user with 1300 rating (118). However, after the increment, the second user still has higher rating than the first user. The rating calculation has the property that if two users do the same hike, then the relative order of their ratings remains the same.

If a user does a hike whose rating is lower than his/her rating?

NO, the system does not decrement the user's rating. Doing a lower rating hike does not mean that user lost all their hiking experience (Sometimes people just want to do relaxing hikes, or they want to hike with less experienced friends). In such scenario, the system still increments user's rating so that they notice the difference. However, the increment is not significant. For example, if a user with rating 1500 does a hike of rating 1200, then their rating will be incremented to 1516. If a user with rating 1800 does the same hike, then increment will be even smaller (1800 → 1813).

Multi day hike:

Since multi day hikes are longer and have more elevation gain compared to single day hikes, their ratings also tend to be higher. The rating update calculation takes into account that the hike was completed in several days and not in a single day. For example, if a user with rating 1200 completes a hike of rating 2500 in a single day, their rating will increase approximately by 400 points (1200 → 1600). On the other hand, if the same user does a 3-day hike with rating 2500, then their rating will only increase by ~100 points (1200 → 1300), which is equivalent to as if the user has done a hike of rating around 1450.

Doing the same route several times:

If a user does the same hike again, then the routing calculation assumes that the user acquires less experience compared to what they would achieve by doing a new route which is equivalent in terms of route rating. Therefore, the rating increment will also reflect that. For example, if a user with rating 1300 does a hike with rating 1500, then their rating will be incremented to 1387. However, if they have already done this hike in the past, then their rating will be incremented to 1343.

Rating Decay:

Some users might have already noticed that sometimes their rating decreases by a couple of points. This is neither a bug in the system nor a side effect of doing an easier hike. We already mentioned that doing an easy hike does not mean that the user lost the experience, and therefore, rating points are not deducted for that.

The only way user can lose rating points is by being inactive. The system counts the inactivity in terms of number of consecutive weeks in which user did not do any hike. This triggers a decay in the rating. That said, the rating of a user will never drop below 700. Since the decay is contiguous, the user will start noticing a decay of 1-2 points after each week of inactivity.

Let us see a couple of examples to observe that this decay is very small.

If a user with rating 1200 does not do any hike for 4 weeks, then their rating will be reduced to 1195. On the other hand, if a user with rating 2100 does not do any hike for 2 weeks, then their rating will be decremented to 2091.