An Empirical Evaluation of Probability Estimation with Neural Networks View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2001-04

AUTHORS

Susan W. Palocsay, Scott P. Stevens, Robert G. Brookshire

ABSTRACT

Recent interest in neural networks by researchers across a wide spectrum of disciplines has provided convincing evidence of their ability to address classification problems. In this article, we consider the issue of evaluating the predictive capability of neural networks when the output values are to be treated as probabilities. We propose the use of a variant of a chi-square statistic, based on the Hosmer–Lemeshow statistic from logistic regression, to measure the goodness-of-fit of neural network models for two-group membership problems. Through experimentation with a large real-world database, we demonstrate the application of this statistic, and examine the effects of varying the number of nodes in the hidden layer on its value. Our empirical results suggest that this statistic can be very useful in identifying significant differences in the probability estimation accuracy of neural network models. More... »

PAGES

48-55

References to SciGraph publications

  • 1983-01. Trauma severity scoring to predict mortality in WORLD JOURNAL OF SURGERY
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s005210170017

    DOI

    http://dx.doi.org/10.1007/s005210170017

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1018731983


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US", 
              "id": "http://www.grid.ac/institutes/grid.258041.a", 
              "name": [
                "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Palocsay", 
            "givenName": "Susan W.", 
            "id": "sg:person.012512771025.25", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012512771025.25"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US", 
              "id": "http://www.grid.ac/institutes/grid.258041.a", 
              "name": [
                "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Stevens", 
            "givenName": "Scott P.", 
            "id": "sg:person.012120605565.18", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012120605565.18"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US", 
              "id": "http://www.grid.ac/institutes/grid.258041.a", 
              "name": [
                "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Brookshire", 
            "givenName": "Robert G.", 
            "id": "sg:person.0614606364.33", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0614606364.33"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/bf01655906", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1000216607", 
              "https://doi.org/10.1007/bf01655906"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2001-04", 
        "datePublishedReg": "2001-04-01", 
        "description": "Recent interest in neural networks by researchers across a wide spectrum of disciplines has provided convincing evidence of their ability to address classification problems. In this article, we consider the issue of evaluating the predictive capability of neural networks when the output values are to be treated as probabilities. We propose the use of a variant of a chi-square statistic, based on the Hosmer\u2013Lemeshow statistic from logistic regression, to measure the goodness-of-fit of neural network models for two-group membership problems. Through experimentation with a large real-world database, we demonstrate the application of this statistic, and examine the effects of varying the number of nodes in the hidden layer on its value. Our empirical results suggest that this statistic can be very useful in identifying significant differences in the probability estimation accuracy of neural network models.", 
        "genre": "article", 
        "id": "sg:pub.10.1007/s005210170017", 
        "inLanguage": "en", 
        "isAccessibleForFree": false, 
        "isPartOf": [
          {
            "id": "sg:journal.1104357", 
            "issn": [
              "0941-0643", 
              "1433-3058"
            ], 
            "name": "Neural Computing and Applications", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "1", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "10"
          }
        ], 
        "keywords": [
          "neural network model", 
          "neural network", 
          "large real-world databases", 
          "network model", 
          "real-world databases", 
          "number of nodes", 
          "classification problem", 
          "hidden layer", 
          "empirical evaluation", 
          "probability estimation", 
          "estimation accuracy", 
          "network", 
          "statistics", 
          "membership problem", 
          "output values", 
          "predictive capability", 
          "chi-square statistics", 
          "empirical results", 
          "problem", 
          "nodes", 
          "model", 
          "recent interest", 
          "estimation", 
          "goodness", 
          "capability", 
          "experimentation", 
          "database", 
          "accuracy", 
          "Hosmer-Lemeshow statistic", 
          "probability", 
          "applications", 
          "fit", 
          "researchers", 
          "issues", 
          "wide spectrum", 
          "disciplines", 
          "values", 
          "spectra", 
          "number", 
          "interest", 
          "evaluation", 
          "layer", 
          "use", 
          "regression", 
          "results", 
          "evidence", 
          "convincing evidence", 
          "variants", 
          "ability", 
          "article", 
          "logistic regression", 
          "effect", 
          "differences", 
          "significant differences"
        ], 
        "name": "An Empirical Evaluation of Probability Estimation with Neural Networks", 
        "pagination": "48-55", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1018731983"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s005210170017"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s005210170017", 
          "https://app.dimensions.ai/details/publication/pub.1018731983"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2022-05-10T09:47", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20220509/entities/gbq_results/article/article_331.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1007/s005210170017"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'


     

    This table displays all metadata directly associated to this object as RDF triples.

    130 TRIPLES      22 PREDICATES      80 URIs      71 LITERALS      6 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s005210170017 schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 schema:author Ncf975a5802604a0a86768670054bb40a
    4 schema:citation sg:pub.10.1007/bf01655906
    5 schema:datePublished 2001-04
    6 schema:datePublishedReg 2001-04-01
    7 schema:description Recent interest in neural networks by researchers across a wide spectrum of disciplines has provided convincing evidence of their ability to address classification problems. In this article, we consider the issue of evaluating the predictive capability of neural networks when the output values are to be treated as probabilities. We propose the use of a variant of a chi-square statistic, based on the Hosmer–Lemeshow statistic from logistic regression, to measure the goodness-of-fit of neural network models for two-group membership problems. Through experimentation with a large real-world database, we demonstrate the application of this statistic, and examine the effects of varying the number of nodes in the hidden layer on its value. Our empirical results suggest that this statistic can be very useful in identifying significant differences in the probability estimation accuracy of neural network models.
    8 schema:genre article
    9 schema:inLanguage en
    10 schema:isAccessibleForFree false
    11 schema:isPartOf N0c6d2fe56f8e419c977a035a31c5f0ec
    12 N3c28776588534d1f89b60fcb26ed3675
    13 sg:journal.1104357
    14 schema:keywords Hosmer-Lemeshow statistic
    15 ability
    16 accuracy
    17 applications
    18 article
    19 capability
    20 chi-square statistics
    21 classification problem
    22 convincing evidence
    23 database
    24 differences
    25 disciplines
    26 effect
    27 empirical evaluation
    28 empirical results
    29 estimation
    30 estimation accuracy
    31 evaluation
    32 evidence
    33 experimentation
    34 fit
    35 goodness
    36 hidden layer
    37 interest
    38 issues
    39 large real-world databases
    40 layer
    41 logistic regression
    42 membership problem
    43 model
    44 network
    45 network model
    46 neural network
    47 neural network model
    48 nodes
    49 number
    50 number of nodes
    51 output values
    52 predictive capability
    53 probability
    54 probability estimation
    55 problem
    56 real-world databases
    57 recent interest
    58 regression
    59 researchers
    60 results
    61 significant differences
    62 spectra
    63 statistics
    64 use
    65 values
    66 variants
    67 wide spectrum
    68 schema:name An Empirical Evaluation of Probability Estimation with Neural Networks
    69 schema:pagination 48-55
    70 schema:productId N34256090086d4e3c9e1563efa208e54e
    71 N3af4056a2d3a4ddba601a14e2fa7dc42
    72 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018731983
    73 https://doi.org/10.1007/s005210170017
    74 schema:sdDatePublished 2022-05-10T09:47
    75 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    76 schema:sdPublisher N12d3cfa2704b47b3ac85086adf28e1b4
    77 schema:url https://doi.org/10.1007/s005210170017
    78 sgo:license sg:explorer/license/
    79 sgo:sdDataset articles
    80 rdf:type schema:ScholarlyArticle
    81 N0c6d2fe56f8e419c977a035a31c5f0ec schema:issueNumber 1
    82 rdf:type schema:PublicationIssue
    83 N12d3cfa2704b47b3ac85086adf28e1b4 schema:name Springer Nature - SN SciGraph project
    84 rdf:type schema:Organization
    85 N34256090086d4e3c9e1563efa208e54e schema:name doi
    86 schema:value 10.1007/s005210170017
    87 rdf:type schema:PropertyValue
    88 N3af4056a2d3a4ddba601a14e2fa7dc42 schema:name dimensions_id
    89 schema:value pub.1018731983
    90 rdf:type schema:PropertyValue
    91 N3c28776588534d1f89b60fcb26ed3675 schema:volumeNumber 10
    92 rdf:type schema:PublicationVolume
    93 N63c0e10fcfc24092b8812f07532a5770 rdf:first sg:person.0614606364.33
    94 rdf:rest rdf:nil
    95 Nb78a1daec40645708bb8dd379c3b12b4 rdf:first sg:person.012120605565.18
    96 rdf:rest N63c0e10fcfc24092b8812f07532a5770
    97 Ncf975a5802604a0a86768670054bb40a rdf:first sg:person.012512771025.25
    98 rdf:rest Nb78a1daec40645708bb8dd379c3b12b4
    99 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    100 schema:name Information and Computing Sciences
    101 rdf:type schema:DefinedTerm
    102 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    103 schema:name Artificial Intelligence and Image Processing
    104 rdf:type schema:DefinedTerm
    105 sg:journal.1104357 schema:issn 0941-0643
    106 1433-3058
    107 schema:name Neural Computing and Applications
    108 schema:publisher Springer Nature
    109 rdf:type schema:Periodical
    110 sg:person.012120605565.18 schema:affiliation grid-institutes:grid.258041.a
    111 schema:familyName Stevens
    112 schema:givenName Scott P.
    113 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012120605565.18
    114 rdf:type schema:Person
    115 sg:person.012512771025.25 schema:affiliation grid-institutes:grid.258041.a
    116 schema:familyName Palocsay
    117 schema:givenName Susan W.
    118 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012512771025.25
    119 rdf:type schema:Person
    120 sg:person.0614606364.33 schema:affiliation grid-institutes:grid.258041.a
    121 schema:familyName Brookshire
    122 schema:givenName Robert G.
    123 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0614606364.33
    124 rdf:type schema:Person
    125 sg:pub.10.1007/bf01655906 schema:sameAs https://app.dimensions.ai/details/publication/pub.1000216607
    126 https://doi.org/10.1007/bf01655906
    127 rdf:type schema:CreativeWork
    128 grid-institutes:grid.258041.a schema:alternateName Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US
    129 schema:name Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US
    130 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...