http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#Head
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo
http://www.nanopub.org/nschema#hasAssertion
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#assertion
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo
http://www.nanopub.org/nschema#hasProvenance
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#provenance
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo
http://www.nanopub.org/nschema#hasPublicationInfo
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#pubinfo
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo
http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://www.nanopub.org/nschema#Nanopublication
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#assertion
http://purl.org/np/RA6f9C7xRc395-jmjLo3hSBCtMALLrwteL6DqWhjyx1fA
http://purl.org/nanopub/x/introduces
http://purl.org/np/RA6f9C7xRc395-jmjLo3hSBCtMALLrwteL6DqWhjyx1fA#figure
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#paragraph
http://purl.org/spar/c4o/hasContent
In the Verify stage of this type of task, the crowd outperformed the precision of the Find stage, achieving values of 0.5510 for the ‘first answer’ setting and 0.8723 with ‘majority voting’. This major improvement on the precision put in evidence the importance of having the a multi-validation pattern like Find-Fix-Verify in which initial errors can be reduced in subsequent iterations. Congruently with the behavior observed in the first workflow, MTurk workers perform well when verifying language-tagged literals. Further- more, the high values of inter-rater agreement con- firms that the crowd is consistently good in this par- ticular scenario. Figure 7b depicts the results of the ‘majority voting’ setting when classifying triples correctly, i.e., true positives (TP) and true negatives (TN), vs. misclassifying triples, i.e. false positives (FP) and false negatives (FN). We can observe that the crowd is exceptionally successful in identifying correct triples that were classified as erroneous in the previous stage (true negatives). This can be confirmed by the high value of accuracy 17 (0.9531) achieved by the crowd in this stage with ‘majority voting’. A closer inspection to the six false positives revealed that in three cases the crowd misclassified triples whose object is a proper noun, for instance, (Tiszaszentimre, name, Tiszaszen- timre@en) and (Ferrari Mythos, label, Ferrari Mythos@de) ; in the other three cases the object of the triple corresponds to a common noun or text in the following languages: Italian, Portuguese, and English, for example, (Book, label, Libro@it) .
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#paragraph
http://www.essepuntato.it/2008/12/pattern#contains
http://purl.org/np/RA6f9C7xRc395-jmjLo3hSBCtMALLrwteL6DqWhjyx1fA#figure
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#paragraph
http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://purl.org/spar/doco/Paragraph
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#provenance
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#assertion
http://www.w3.org/ns/prov#hadPrimarySource
http://dx.doi.org/10.3233/SW-160239
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#assertion
http://www.w3.org/ns/prov#wasAttributedTo
https://orcid.org/0000-0003-0530-4305
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo#pubinfo
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo
http://purl.org/dc/terms/created
2019-09-20T18:05:11+01:00
http://purl.org/np/RAtwDxBF1MJoX9AzkW5vkAFZ3mcprtiP6-MGybEB2Zevo
http://purl.org/pav/createdBy
https://orcid.org/0000-0002-7114-6459