When we classify texts to declare that it contains a relation to an ethical issue in software it is important to note that the users’ perspective may vary from ours. We must therefore base our annotations on the types of EIS that we have learned of from others in previous research.
There are several components that lead to a positive classification as EIS.
In this study we only focus on ethical issues that relate to the software applications that the Reddit posts were written about. The data you are dealing with should show which app the text is supposed to relate to. If the post does not relate to this application, ask yourself if it relates to a different application that is related to this one. There are various services that are embedded in each other, such as online advertising platforms, cloud services, and most recently, AI chatbot customer service.
See if it fits our definition!
On Reddit, we most often observe people speaking out against an ethical issue. Redditors express their ethical concerns to talk with others about their worries.
Rarely, Redditors create posts that directly dismisses another group's worries, although there are examples.
If a text speaks about a specific topic that we know is seen as an ethical issue (see our EIS types and subtypes), but the author is not concerned about it, it should still be given a TRUE label for containing an EIS. An example would be this post that speaks favorable about ad tracking technology, which is a software aspect that is at odds with the ethics of other people. See Privacy.
Look at our taxonomy, all the types that we have identified are listed in the sidebar.
In the next step you will have to annotate the type of EIS. The texts may not fit neatly into the existing categories. If you think that from the author's perspective more than one type applies write all the types down that apply. Text that does not fit known classes should be classified as “other”.
It might help to think about stance already at this stage. Would this person say that it is an issue to them and are they worried? They would have an 'against' stance on the issue. Is it a neutral description of a specific issue? It would be a 'neutral' stance. Does the person express that they are in favor of something happening that others take issue with? Then it might be an 'in_favor' stance towards an issue. The last case can only be detected if they acknowledge in their text that this is an issue to some, of if we previous knowledge about it within our taxonomy (Such as the sub-types given in Privacy).
If you believe that either of these three are the case, then it is highly likely that it is an example of an ethical issue.
Check out each of the ethical issues in software that are listed in the sidebar! Each page contains a definition and one or more examples.