http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#Head
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk
http://www.nanopub.org/nschema#hasAssertion
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk
http://www.nanopub.org/nschema#hasProvenance
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#provenance
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk
http://www.nanopub.org/nschema#hasPublicationInfo
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#pubinfo
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk
http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://www.nanopub.org/nschema#Nanopublication
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#paragraph
http://purl.org/spar/c4o/hasContent
Accordingly, the results of both Find stages (ex- pert and workers) – in the form of sets of triples iden- tified as incorrect, marked with the respective errors – were fed into a subsequent Verify step, carried out by MTurk workers (Section 4.3). The task consisted solely of the rating of a formerly indicated quality is- sue for a triple as correctly or wrongly assigned. This Verify step was, in fact, able to improve the preci- sion of both Find stages substantially. In particular, the experts’ Find stage results could be improved to precision levels of around 0.9 in the Verify stage for two error types which showed to score much lower for an expert-only Find approach. The worker-worker Find-Verify strategy yielded also better results than the Find-only worker approach, and for one error type even reached slightly better precision than the expert-worker model. All in all, we show that (i) a Find-Verify combination of experts and lay users is likely to pro- duce the best results, but that (ii) they are not superior to expert-only evaluation in all cases. We demonstrate also that (iii) lay users-only Find-Verify approaches can be a viable alternative for detection of LD qual- ity issues if experts are not available and that they cer- tainly outperform Find-only lay user workflows.
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#paragraph
http://www.w3.org/1999/02/22-rdf-syntax-ns#type
http://purl.org/spar/doco/Paragraph
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#provenance
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion
http://www.w3.org/ns/prov#hadPrimarySource
http://dx.doi.org/10.3233/SW-160239
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion
http://www.w3.org/ns/prov#wasAttributedTo
https://orcid.org/0000-0003-0530-4305
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#pubinfo
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk
http://purl.org/dc/terms/created
2019-11-10T12:34:11+01:00
http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk
http://purl.org/pav/createdBy
https://orcid.org/0000-0002-7114-6459