[Usability] Why I don't like GNOME Software's rating system

[rugk <rugk posteo de>]

Hi,
I am writing to this mailing list, explaining my issue(s) I have with 
how ODRS is designed, as my original bug request in 
https://bugzilla.gnome.org/show_bug.cgi?id=783252 apparently did not got 
enough attention. Propbably mailing lists are more suitable for this 
kind of discussions.

FYI I've also tried to subscribe to this list on https://mail.gnome.org, 
but it always claims my captcha is wrong. So if you could subscribe me 
or just add my mail here when replying to this mail, please do so.
If you think I wrote to the wrong list (could not find one about privacy 
or GNOME Software), please repost it in the correct one.

---

First let me say that I am speaking out of the perspective of a (nooby) 
privacy-conscious GNOME software user. I'll noticed many privacy issues 
when using it and some more when reading the description on 
https://odrs.gnome.org/. I see your decisions were well-intentioned, but 
some are just bad for the privacy of your users.
I'll describe my personal experience here, to make more understandable 
what issues I encountered. Doing so I will use some ****** words just to 
make it more vivid, so please don't freak out because of them. All I 
want to be is constructive, I would not create this issue otherwise. (I 
would rather rant in a blog or in social media in that case…)

In short: This is a German/nooby/privacy-aware/first-time experience 
using GNOME software. Version 3.22.7 BTW.
Attention: This is a long read. Take some time.

== Problems ==
1. The unknowing user…

– The problem:
A new user does not know what's submitted. As a new user of GNOME 
software I see that I can evaluate apps. That's nice, but as a new user 
I also like to know what will be submitted. And the interface just gives 
mo *no* clues about this. I can only speculate
And, of course, users should not have to look online how the mechanism 
works. That must be clear in the software.

First of all **there is no link to any privacy policy**. And there are 
many other things I wondered about and did not know before (& sometimes 
even after submitting a rating):
1. Which username will be used? The "usual" one (/home/<username>), the 
full name I give when creating an account? Or will I be able to choose 
the username? (I would prefer the latter, BTW.)
2. What other identifiers are submitted? My country? My OS/distro? My 
language?
3. Will I be able to edit my review?
4. Will I be able to delete it, in case I regret my review later?
5. Some may even ask: Will this published? (i.e. it is a good idea to 
just note there: "This will be made public."

As I should notice later, also on https://odrs.gnome.org/ there is no 
link to your software ratings.

– How to solve:
Either when first making a review or in the review window generally, add 
a small text with a summary, such as: "Note that this review will be 
made public including your username and some information about your 
machine. For more information click here. <link to privacy policy>"

2. User names
– The problem:
So after the first review the user may realize,: "Oh f***, they 
submitted the full name of my Linux account." Seriously, that's not only 
unexpected, but bad. I would have expected you to submit the usual linux 
user name, but to submit the whole name is not so nice, especially as 
many, many users are used to use their real first name (and some maybe 
also second name) as their user account. (You can confirm this by 
looking at the reviews…)

So, do you want to know what I thought?
I thought: "Okay, the GNOME guys are good people, so maybe they don't 
show it public, only uploaded my short user name (or an ID or so) and 
now they only parse it locally and show the local user name, they 
found." I thought this might be useful, as local users using the same 
system could then notice I was the one, who made the review.

But as I should find out, I was wrong.

– How to solve:
Usernames are a difficult topic. I, however, cannot accept that my local 
username is used there without at least asking me before. You have to 
know that's also quite contrary to the system that users are used to in 
all other review systems and similar "web services" are used to. Usually 
they always have to sign up and can at least determinate their user name 
by themselves.They may use different user names to separate different 
identities. They may never mention their real name online. They may of 
course also mention their full name if they like.
The point is: The user has to decide, what he/she wants to share. In 
Germany, we call this informational self-determination. (Just search for 
it…)

That's why I propose to let the user decide it. Have a way for the user 
to change their user name, which should be reflected in all their 
reviews. When doing their first review, you can e.g. ask them what user 
name they like to have. You can pre-fill the input field with the full 
user name, you currently use and users may choose to use that name. If 
so, that is their choice!
In any case, you'll save that and just re-use that name until the user 
changes it. When doing so it should be updated in all existing reviews, 
IMHO.

3. The "hidden" information & the backend system
– The problem:
What I did not know was, that much more information was submitted than 
just my user name. Okay, my distro and my language setting are okay for 
me, and I would have expected these. (Although it should have been 
mentioned in a privacy policy, at least, see #1).
But I did not expect to submit a SHA-1 hash…
When I read up on this feature (After I found the site 
https://odrs.gnome.org/, which was more accidental and not easy to 
find.) I could finally see what is submitted:
-> "we're using a SHA1 hash of the machine-id and the UNIX username 
along with a salt"

Okay, at least a salt. The problem is I cannot see it anywhere in the 
ratings, so is this one static salt for all of the ratings or – as it 
would be correct – one for each rating?
In the end, it does not matter, because you can likely brute-force it 
anyway. (see below)
So let's look at the content. Now, in addition to the full name, we have 
the "short username" included here. This means with some brute-forcing 
one can connect these names, which may be bad as some users might choose 
"pseudonym" as their short name and (because they think this name is 
only showed in some specific places, locally) use their real name as a 
full name. When an attacker does so, this might essentially completely 
destroy a user's privacy as the pseudonym could be used on other 
websites too.

I don't know whether the machine ID could be useful, maybe it is when 
other software uses it to publish more information in a similar way, but 
in any case one can at least find out how many users are using one 
device or are, at least, rating there.

– How to solve:
The best would be to not use this data. This is however not practical, 
so you may follow the advice below.

4. A special note about SHA-1
– The problem:
You should not use SHA-1 anymore. You may have seen 
https://shattered.it/, you may have noticed it has been depreciated by 
NIST in 2011, you may have heard theoretical attacks already appeared in 
2005.
You may still say, for this use it is okay. Yes, it may be, if the 
machine ID is really random (which I don't know if that is the case) and 
it may be even harder to brute-force if the user name is also quite long 
(which you, however, cannot assume). When an attacker however e.g. knows 
the machine ID it may be easy to brute-force.

Also for the user_key it "should be impossible to generate a user_key 
from a user_id without first requesting the reviews from the server". 
Well, with shattered (& future attacks on SHA-1) it may be possible.

– How to solve:
In general the algorithm should just not be used anymore. Switch to 
SHA-256, at least.

However, the best idea would be to switch to a proper algorithm designed 
for your use case. Because basically this machine ID and username is a 
"password". A thing, only one user "knows" (or "has", in this case) and, 
which should be slow to brute-force and some milliseconds to wait while 
hashing it is not bad. For these there are specially designed hashes, 
such as bcrypt, scrypt or – the new winner 
(https://en.wikipedia.org/wiki/Password_Hashing_Competition) – ARGON2. 
These hashes are very slow (in contrast to SHA-256, which is optimized 
for speed) and thus difficult to brute-force, making sure no one can 
ever access the machine ID or username in the ID. In general see 
https://crackstation.net/hashing-security.htm for that. It is exactly 
what you need.

Of course you need to stay backward-compatible, so for some time you can 
accept the old hashes, but switch to the new hashes in the application. 
Later you can then disallow the old hashes to be used, so that now only 
the "secure reviews" can be written.

5. A general word on "anonymous"
– The problem:
Now, on https://odrs.gnome.org/, I read: "Designing an anonymous service 
is hard". Wait, wait…
As we learned, you submit full usernames and some obfuscated machine IDs 
and short user names. This is not anonymous, at all. That's 
pseudonymity.
See https://internet-anonymity.com/internet-anonymity-vs-pseudonymity/ 
for a quick explanation.

Even without the full user name it would be pseudonymous as each user 
has an ID (the SHA-1 hash shown public).

– How to solve:
Never lie to the user. Thankfully, this was only a doc page, so it is 
not in your privacy policy (which is not there, anyway…), but it is 
still not good. Especially when writing this under the headline 
"privacy".

So change the wording and call it "pseudonymous".

5. The deletion system is a farce.
– The problem:
After seeing this all, I decided to delete my reviews I already made. I 
was reassured when I saw this is possible (and finding them in this 
great list online was easy), so I could delete all of them. Now, 
everything is all right? Right? No.

Because they are still there, online. Anyone can still access them. Once 
reviewed, once made a mistake of not changing your username (because one 
did not know the impact, see #1) and your done. Irreversible.
That's crazy. Only a small "deleted at" date is added to the entry. It 
is still accessible online. If the user was not me (who checked that on 
the website), he/she would not even have known that. They would have a 
false sense of security, or, here, privacy, and may think all data is 
gone. But wrong.

– How to solve:
Please, please, allow users to really remove their own reviews. Why do 
you still need to store them after they decided to delete them? If you 
want to have some "restore functionality", then maybe store them a few 
days and then automatically delete them, but do not leave them there. 
This exploits the trust, your users out into your service (or this exact 
one button, there). "Delete" means "delete". In this case that's very 
clear and also other review platforms such as AMO (addons.mozilla.org) 
allow this.

Speaking about AMO another nice way to help users to stay in control of 
their data (aka reviews) would be to have a page, where they can see all 
of their reviews. That's also a standard feature of all review 
platforms.
This allows them to review their reviews (pun not intended) and delete 
or edit them if necessary. (Editing is not yet possible, but basically 
one could also just delete and add a new review, so I don't know whether 
you'll implement this in the future.)
It's also good to see in case you want to change a rating.

== But why? ==
Before you say this would not matter ("You have nothing to hide-like"), 
let me make a fictional example. If there would be some porn apps in the 
app store (don't know whether this is allowed or so, this does not 
matter here anyway), then a user will likely not like it to be exposed 
when creating a review there. The user would not want his/her user name 
to be published without consent, neither any additional information, 
which could identify them. At least they should know about the latter. 
You claim to have an anonymous rate system. This is false. And this user 
would regret it. But even when doing so, deleting the review would not 
help.
That's bad, very bad…
Or, maybe, in some countries, a software review of a "hacking tool" 
maybe can get you killed?

All in all: The current state is really a no-go. These are five big 
points you should tackle. Personally I would have rather signed up for a 
real user account there*, where I can choose, which information I 
provide. Personally I am a guy, who would rather have not such a "fake 
anonymous" review system than just a usual evaluation system, where I 
have everything under my control. And the big promise of FLOSS software 
is that the user can have everything under their own control. Don't 
destroy this promise!

* Also using "real" account, would help when users are distro-hoppers or 
so and frequently install new distros or just set their system up 
another time. Currently, they loose "access" to all their previous 
reviews with this system.

Best regards,
rugk

----
I am temporarily testing PGP for this mail address.

Fingerprint: 7046 C1B2 8644 9EAF 9F3F F5C1 8F16 2AE4 4088 F1BE
Key: 
https://keys.mailvelope.com/pks/lookup?op=get&search=0x8F162AE44088F1BE