Multi-criteria Fuzzy Ordinal Peer Assessment for MOOCs View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2019

AUTHORS

Nicola Capuano , Santi Caballé

ABSTRACT

Due to the high number of students enrolled and the relatively small number of available tutors, the assessment of complex assignments is deemed as one of the most critical tasks in Massive Open On-line Courses (MOOCs). Peer assessment is becoming an increasingly popular tool to face this problem and many approaches have been proposed so far to make its outcomes more reliable. A promising approach is FOPA (Fuzzy Ordinal Peer Assessment) that adopts and integrates models coming from Fuzzy Set Theory and Group Decision Making. In this paper we propose a FOPA extension supporting multi-criteria assessment based on rubrics. Students are asked to rank a small number of peer submissions against specified criteria, provided rankings are then transformed in fuzzy preference relations, expanded to obtain missing values and aggregated to establish a global ranking between students’ works with respect to each criterion and globally. The absolute grades of all submissions are then calculated. More... »

PAGES

373-383

References to SciGraph publications

  • 2018. Application of Fuzzy Ordinal Peer Assessment in Formative Evaluation in ADVANCES ON P2P, PARALLEL, GRID, CLOUD AND INTERNET COMPUTING
  • Book

    TITLE

    Advances in Intelligent Networking and Collaborative Systems

    ISBN

    978-3-319-98556-5
    978-3-319-98557-2

    Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/978-3-319-98557-2_34

    DOI

    http://dx.doi.org/10.1007/978-3-319-98557-2_34

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1106329554


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information Systems", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "University of Salerno", 
              "id": "https://www.grid.ac/institutes/grid.11780.3f", 
              "name": [
                "University of Salerno"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Capuano", 
            "givenName": "Nicola", 
            "id": "sg:person.011324165665.56", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011324165665.56"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Open University of Catalonia", 
              "id": "https://www.grid.ac/institutes/grid.36083.3e", 
              "name": [
                "Universitat Oberta de Catalunya"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Caball\u00e9", 
            "givenName": "Santi", 
            "id": "sg:person.015625351110.52", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015625351110.52"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "https://doi.org/10.1080/02602930902862859", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1003759883"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/0165-0114(93)90194-m", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1011541460"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/0165-0114(93)90194-m", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1011541460"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.na.2008.11.087", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1021920063"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.ejor.2006.08.032", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1023562851"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.cie.2007.05.001", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1024633024"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.edurev.2007.05.002", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1027595384"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/s0165-0114(96)00339-9", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1042363149"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/0898-1221(83)90013-5", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1045040132"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/2623330.2623654", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1047233469"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1002/int.20262", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1053472016"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tlt.2016.2565476", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061665521"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1142/s0219622009003417", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1063000979"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.3991/ijet.v11i07.5878", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1071809661"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.5210/fm.v18i5.4350", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1072709948"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tfuzz.2017.2744605", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1091346839"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-69835-9_52", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1092499860", 
              "https://doi.org/10.1007/978-3-319-69835-9_52"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/3pgcic.2015.7", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094071328"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/fie.2003.1264740", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095188341"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1142/p505", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1099057489"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1002/int.21997", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1101845666"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1002/9780470974032", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1106821027"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2019", 
        "datePublishedReg": "2019-01-01", 
        "description": "Due to the high number of students enrolled and the relatively small number of available tutors, the assessment of complex assignments is deemed as one of the most critical tasks in Massive Open On-line Courses (MOOCs). Peer assessment is becoming an increasingly popular tool to face this problem and many approaches have been proposed so far to make its outcomes more reliable. A promising approach is FOPA (Fuzzy Ordinal Peer Assessment) that adopts and integrates models coming from Fuzzy Set Theory and Group Decision Making. In this paper we propose a FOPA extension supporting multi-criteria assessment based on rubrics. Students are asked to rank a small number of peer submissions against specified criteria, provided rankings are then transformed in fuzzy preference relations, expanded to obtain missing values and aggregated to establish a global ranking between students\u2019 works with respect to each criterion and globally. The absolute grades of all submissions are then calculated.", 
        "editor": [
          {
            "familyName": "Xhafa", 
            "givenName": "Fatos", 
            "type": "Person"
          }, 
          {
            "familyName": "Barolli", 
            "givenName": "Leonard", 
            "type": "Person"
          }, 
          {
            "familyName": "Gregu\u0161", 
            "givenName": "Michal", 
            "type": "Person"
          }
        ], 
        "genre": "chapter", 
        "id": "sg:pub.10.1007/978-3-319-98557-2_34", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": false, 
        "isPartOf": {
          "isbn": [
            "978-3-319-98556-5", 
            "978-3-319-98557-2"
          ], 
          "name": "Advances in Intelligent Networking and Collaborative Systems", 
          "type": "Book"
        }, 
        "name": "Multi-criteria Fuzzy Ordinal Peer Assessment for MOOCs", 
        "pagination": "373-383", 
        "productId": [
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/978-3-319-98557-2_34"
            ]
          }, 
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "4793ff9f3839eb191374b0f81a690e83596b910ebfb83a813b1576a9343dddf7"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1106329554"
            ]
          }
        ], 
        "publisher": {
          "location": "Cham", 
          "name": "Springer International Publishing", 
          "type": "Organisation"
        }, 
        "sameAs": [
          "https://doi.org/10.1007/978-3-319-98557-2_34", 
          "https://app.dimensions.ai/details/publication/pub.1106329554"
        ], 
        "sdDataset": "chapters", 
        "sdDatePublished": "2019-04-15T21:41", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8690_00000605.jsonl", 
        "type": "Chapter", 
        "url": "http://link.springer.com/10.1007/978-3-319-98557-2_34"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-98557-2_34'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-98557-2_34'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-98557-2_34'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-98557-2_34'


     

    This table displays all metadata directly associated to this object as RDF triples.

    149 TRIPLES      23 PREDICATES      48 URIs      20 LITERALS      8 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/978-3-319-98557-2_34 schema:about anzsrc-for:08
    2 anzsrc-for:0806
    3 schema:author Nb54034ae808c4d19a7662df5330daab2
    4 schema:citation sg:pub.10.1007/978-3-319-69835-9_52
    5 https://doi.org/10.1002/9780470974032
    6 https://doi.org/10.1002/int.20262
    7 https://doi.org/10.1002/int.21997
    8 https://doi.org/10.1016/0165-0114(93)90194-m
    9 https://doi.org/10.1016/0898-1221(83)90013-5
    10 https://doi.org/10.1016/j.cie.2007.05.001
    11 https://doi.org/10.1016/j.edurev.2007.05.002
    12 https://doi.org/10.1016/j.ejor.2006.08.032
    13 https://doi.org/10.1016/j.na.2008.11.087
    14 https://doi.org/10.1016/s0165-0114(96)00339-9
    15 https://doi.org/10.1080/02602930902862859
    16 https://doi.org/10.1109/3pgcic.2015.7
    17 https://doi.org/10.1109/fie.2003.1264740
    18 https://doi.org/10.1109/tfuzz.2017.2744605
    19 https://doi.org/10.1109/tlt.2016.2565476
    20 https://doi.org/10.1142/p505
    21 https://doi.org/10.1142/s0219622009003417
    22 https://doi.org/10.1145/2623330.2623654
    23 https://doi.org/10.3991/ijet.v11i07.5878
    24 https://doi.org/10.5210/fm.v18i5.4350
    25 schema:datePublished 2019
    26 schema:datePublishedReg 2019-01-01
    27 schema:description Due to the high number of students enrolled and the relatively small number of available tutors, the assessment of complex assignments is deemed as one of the most critical tasks in Massive Open On-line Courses (MOOCs). Peer assessment is becoming an increasingly popular tool to face this problem and many approaches have been proposed so far to make its outcomes more reliable. A promising approach is FOPA (Fuzzy Ordinal Peer Assessment) that adopts and integrates models coming from Fuzzy Set Theory and Group Decision Making. In this paper we propose a FOPA extension supporting multi-criteria assessment based on rubrics. Students are asked to rank a small number of peer submissions against specified criteria, provided rankings are then transformed in fuzzy preference relations, expanded to obtain missing values and aggregated to establish a global ranking between students’ works with respect to each criterion and globally. The absolute grades of all submissions are then calculated.
    28 schema:editor N96612fe40c914f87bc5991d9568b83c9
    29 schema:genre chapter
    30 schema:inLanguage en
    31 schema:isAccessibleForFree false
    32 schema:isPartOf Nb70a9faa5adf41dfb6f7e40214c04eac
    33 schema:name Multi-criteria Fuzzy Ordinal Peer Assessment for MOOCs
    34 schema:pagination 373-383
    35 schema:productId N34f3b3ad80984d0fb6669ea41bab90b4
    36 Nae21d5109b5e4970a304bab62f20c0ab
    37 Nf27ad70d65c945c7a96e8cc89a8a065b
    38 schema:publisher Nd8199f285cff40578394b9490e44422e
    39 schema:sameAs https://app.dimensions.ai/details/publication/pub.1106329554
    40 https://doi.org/10.1007/978-3-319-98557-2_34
    41 schema:sdDatePublished 2019-04-15T21:41
    42 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    43 schema:sdPublisher Nda50f47397c3490dbc4adf2274b75234
    44 schema:url http://link.springer.com/10.1007/978-3-319-98557-2_34
    45 sgo:license sg:explorer/license/
    46 sgo:sdDataset chapters
    47 rdf:type schema:Chapter
    48 N07a84b40322c4715a2b11ddbcb76af21 schema:familyName Greguš
    49 schema:givenName Michal
    50 rdf:type schema:Person
    51 N34f3b3ad80984d0fb6669ea41bab90b4 schema:name readcube_id
    52 schema:value 4793ff9f3839eb191374b0f81a690e83596b910ebfb83a813b1576a9343dddf7
    53 rdf:type schema:PropertyValue
    54 N3daea558750b470c81d10000be0bc183 rdf:first N07a84b40322c4715a2b11ddbcb76af21
    55 rdf:rest rdf:nil
    56 N41fb001f5d184b849c93f303f804a2ad schema:familyName Xhafa
    57 schema:givenName Fatos
    58 rdf:type schema:Person
    59 N96612fe40c914f87bc5991d9568b83c9 rdf:first N41fb001f5d184b849c93f303f804a2ad
    60 rdf:rest N9c03d5a755344c18ae24908723d28cfc
    61 N9c03d5a755344c18ae24908723d28cfc rdf:first Na1d2f85833094130bd45f8bab2def368
    62 rdf:rest N3daea558750b470c81d10000be0bc183
    63 N9c0dde18e36c4f7bb2b8eef973ccf3ef rdf:first sg:person.015625351110.52
    64 rdf:rest rdf:nil
    65 Na1d2f85833094130bd45f8bab2def368 schema:familyName Barolli
    66 schema:givenName Leonard
    67 rdf:type schema:Person
    68 Nae21d5109b5e4970a304bab62f20c0ab schema:name doi
    69 schema:value 10.1007/978-3-319-98557-2_34
    70 rdf:type schema:PropertyValue
    71 Nb54034ae808c4d19a7662df5330daab2 rdf:first sg:person.011324165665.56
    72 rdf:rest N9c0dde18e36c4f7bb2b8eef973ccf3ef
    73 Nb70a9faa5adf41dfb6f7e40214c04eac schema:isbn 978-3-319-98556-5
    74 978-3-319-98557-2
    75 schema:name Advances in Intelligent Networking and Collaborative Systems
    76 rdf:type schema:Book
    77 Nd8199f285cff40578394b9490e44422e schema:location Cham
    78 schema:name Springer International Publishing
    79 rdf:type schema:Organisation
    80 Nda50f47397c3490dbc4adf2274b75234 schema:name Springer Nature - SN SciGraph project
    81 rdf:type schema:Organization
    82 Nf27ad70d65c945c7a96e8cc89a8a065b schema:name dimensions_id
    83 schema:value pub.1106329554
    84 rdf:type schema:PropertyValue
    85 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    86 schema:name Information and Computing Sciences
    87 rdf:type schema:DefinedTerm
    88 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
    89 schema:name Information Systems
    90 rdf:type schema:DefinedTerm
    91 sg:person.011324165665.56 schema:affiliation https://www.grid.ac/institutes/grid.11780.3f
    92 schema:familyName Capuano
    93 schema:givenName Nicola
    94 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011324165665.56
    95 rdf:type schema:Person
    96 sg:person.015625351110.52 schema:affiliation https://www.grid.ac/institutes/grid.36083.3e
    97 schema:familyName Caballé
    98 schema:givenName Santi
    99 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015625351110.52
    100 rdf:type schema:Person
    101 sg:pub.10.1007/978-3-319-69835-9_52 schema:sameAs https://app.dimensions.ai/details/publication/pub.1092499860
    102 https://doi.org/10.1007/978-3-319-69835-9_52
    103 rdf:type schema:CreativeWork
    104 https://doi.org/10.1002/9780470974032 schema:sameAs https://app.dimensions.ai/details/publication/pub.1106821027
    105 rdf:type schema:CreativeWork
    106 https://doi.org/10.1002/int.20262 schema:sameAs https://app.dimensions.ai/details/publication/pub.1053472016
    107 rdf:type schema:CreativeWork
    108 https://doi.org/10.1002/int.21997 schema:sameAs https://app.dimensions.ai/details/publication/pub.1101845666
    109 rdf:type schema:CreativeWork
    110 https://doi.org/10.1016/0165-0114(93)90194-m schema:sameAs https://app.dimensions.ai/details/publication/pub.1011541460
    111 rdf:type schema:CreativeWork
    112 https://doi.org/10.1016/0898-1221(83)90013-5 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045040132
    113 rdf:type schema:CreativeWork
    114 https://doi.org/10.1016/j.cie.2007.05.001 schema:sameAs https://app.dimensions.ai/details/publication/pub.1024633024
    115 rdf:type schema:CreativeWork
    116 https://doi.org/10.1016/j.edurev.2007.05.002 schema:sameAs https://app.dimensions.ai/details/publication/pub.1027595384
    117 rdf:type schema:CreativeWork
    118 https://doi.org/10.1016/j.ejor.2006.08.032 schema:sameAs https://app.dimensions.ai/details/publication/pub.1023562851
    119 rdf:type schema:CreativeWork
    120 https://doi.org/10.1016/j.na.2008.11.087 schema:sameAs https://app.dimensions.ai/details/publication/pub.1021920063
    121 rdf:type schema:CreativeWork
    122 https://doi.org/10.1016/s0165-0114(96)00339-9 schema:sameAs https://app.dimensions.ai/details/publication/pub.1042363149
    123 rdf:type schema:CreativeWork
    124 https://doi.org/10.1080/02602930902862859 schema:sameAs https://app.dimensions.ai/details/publication/pub.1003759883
    125 rdf:type schema:CreativeWork
    126 https://doi.org/10.1109/3pgcic.2015.7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094071328
    127 rdf:type schema:CreativeWork
    128 https://doi.org/10.1109/fie.2003.1264740 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095188341
    129 rdf:type schema:CreativeWork
    130 https://doi.org/10.1109/tfuzz.2017.2744605 schema:sameAs https://app.dimensions.ai/details/publication/pub.1091346839
    131 rdf:type schema:CreativeWork
    132 https://doi.org/10.1109/tlt.2016.2565476 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061665521
    133 rdf:type schema:CreativeWork
    134 https://doi.org/10.1142/p505 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099057489
    135 rdf:type schema:CreativeWork
    136 https://doi.org/10.1142/s0219622009003417 schema:sameAs https://app.dimensions.ai/details/publication/pub.1063000979
    137 rdf:type schema:CreativeWork
    138 https://doi.org/10.1145/2623330.2623654 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047233469
    139 rdf:type schema:CreativeWork
    140 https://doi.org/10.3991/ijet.v11i07.5878 schema:sameAs https://app.dimensions.ai/details/publication/pub.1071809661
    141 rdf:type schema:CreativeWork
    142 https://doi.org/10.5210/fm.v18i5.4350 schema:sameAs https://app.dimensions.ai/details/publication/pub.1072709948
    143 rdf:type schema:CreativeWork
    144 https://www.grid.ac/institutes/grid.11780.3f schema:alternateName University of Salerno
    145 schema:name University of Salerno
    146 rdf:type schema:Organization
    147 https://www.grid.ac/institutes/grid.36083.3e schema:alternateName Open University of Catalonia
    148 schema:name Universitat Oberta de Catalunya
    149 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...