http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#Head http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk http://www.nanopub.org/nschema#hasAssertion http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk http://www.nanopub.org/nschema#hasProvenance http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#provenance http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk http://www.nanopub.org/nschema#hasPublicationInfo http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#pubinfo http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://www.nanopub.org/nschema#Nanopublication http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#paragraph http://purl.org/spar/c4o/hasContent Accordingly, the results of both Find stages (ex- pert and workers) – in the form of sets of triples iden- tified as incorrect, marked with the respective errors – were fed into a subsequent Verify step, carried out by MTurk workers (Section 4.3). The task consisted solely of the rating of a formerly indicated quality is- sue for a triple as correctly or wrongly assigned. This Verify step was, in fact, able to improve the preci- sion of both Find stages substantially. In particular, the experts’ Find stage results could be improved to precision levels of around 0.9 in the Verify stage for two error types which showed to score much lower for an expert-only Find approach. The worker-worker Find-Verify strategy yielded also better results than the Find-only worker approach, and for one error type even reached slightly better precision than the expert-worker model. All in all, we show that (i) a Find-Verify combination of experts and lay users is likely to pro- duce the best results, but that (ii) they are not superior to expert-only evaluation in all cases. We demonstrate also that (iii) lay users-only Find-Verify approaches can be a viable alternative for detection of LD qual- ity issues if experts are not available and that they cer- tainly outperform Find-only lay user workflows. http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#paragraph http://www.w3.org/1999/02/22-rdf-syntax-ns#type http://purl.org/spar/doco/Paragraph http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#provenance http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion http://www.w3.org/ns/prov#hadPrimarySource http://dx.doi.org/10.3233/SW-160239 http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#assertion http://www.w3.org/ns/prov#wasAttributedTo https://orcid.org/0000-0003-0530-4305 http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk#pubinfo http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk http://purl.org/dc/terms/created 2019-11-10T12:34:11+01:00 http://purl.org/np/RAarh8wySNE2FwQg3Or82OWP1VXVlYe-0q64eBrQfWpUk http://purl.org/pav/createdBy https://orcid.org/0000-0002-7114-6459