Differences

This shows you the differences between two versions of the page.

--- eis:resources:annotation_guide [2025/02/04 13:37] – removed - external edit (Unknown date) 127.0.0.1
+++ eis:resources:annotation_guide [2025/02/05 12:13] (current) – tom
@@ Line 1: / Line 1: @@
+====== Annotation Guide ======
+<WRAP group>
+<WRAP half column>
+===== Introduction =====
+Our study aims at identifying stances towards ethical issues in software.\\
+Thank you for helping us in our study. Your task is to read user posts from Reddit and label them according to a given set of codes. This guide describes the instructions that you should follow carefully to conduct the task successfully.
+You should have access to a link together with this guide, pointing at the data sheets which you use to complete this task.
+<wrap hi>Please, have a look at the [[eis:resources:eis_classification|EIS classification]] section.</wrap>
+===== Labeling posts =====
+The task is to read the posts in the Excel sheet one by one. For each Reddit post, we need to:
+<WRAP center round box blue 90%>
+<WRAP left round box 35px white>
+</WRAP>
+Read the Reddit post closely, both title and body. Decide if the post addresses an [[eis:home|ethical issue in software (EIS)]].
+If it does not, you label it with FALSE in the ‘has_eis’ column and move on to the next post. If it does, label it with TRUE and continue to the next step.
+</WRAP>
+<WRAP center box purple 90%>
+<WRAP left round box 35px white>
+</WRAP>
+Decide which type of ethical issue the post addresses. Please see the full list of ethical issues in software with examples [[eis:home|here]] and in the sidebar. You can select one or more types in the dropdown list of column ‘eis_types’. If none apply, select other.
+</WRAP>
+<WRAP center round box pink 90%>
+<WRAP left round box 35px white>
+</WRAP>
+Consider the perspective of the author for the last step. Which side of the issue would they take? If there is not enough information to make this decision, label it as ‘neutral’ in the stance column. Otherwise decide if the author takes side with those that see an ethical issue in software (against stance towards EIS), or if they are on the opposite side of the issue (in favor stance towards EIS). See more details and examples in the Classification section.
+<WRAP left round box 35px white>
++
+</WRAP>
+\\
+If you chose either in favor or against, try to select and copy/paste the part of the text that contributed most to your decision into the deciding_factor column.
+</WRAP>
+</WRAP>
+<WRAP half column>
+===== Short Glossary =====
+{{tabembed> :glossary:ethics|Ethics, :glossary:eis|Ethical issue in Software, :glossary:reddit|Reddit, :glossary:software|Software, :glossary:stance|Stance}}
+</WRAP>
+</WRAP>