An Empirical Evaluation of Probability Estimation with Neural Networks View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2001-04

AUTHORS

Susan W. Palocsay, Scott P. Stevens, Robert G. Brookshire

ABSTRACT

Recent interest in neural networks by researchers across a wide spectrum of disciplines has provided convincing evidence of their ability to address classification problems. In this article, we consider the issue of evaluating the predictive capability of neural networks when the output values are to be treated as probabilities. We propose the use of a variant of a chi-square statistic, based on the Hosmer–Lemeshow statistic from logistic regression, to measure the goodness-of-fit of neural network models for two-group membership problems. Through experimentation with a large real-world database, we demonstrate the application of this statistic, and examine the effects of varying the number of nodes in the hidden layer on its value. Our empirical results suggest that this statistic can be very useful in identifying significant differences in the probability estimation accuracy of neural network models. More... »

PAGES

48-55

References to SciGraph publications

  • 1983-01. Trauma severity scoring to predict mortality in WORLD JOURNAL OF SURGERY
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s005210170017

    DOI

    http://dx.doi.org/10.1007/s005210170017

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1018731983


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US", 
              "id": "http://www.grid.ac/institutes/grid.258041.a", 
              "name": [
                "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Palocsay", 
            "givenName": "Susan W.", 
            "id": "sg:person.012512771025.25", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012512771025.25"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US", 
              "id": "http://www.grid.ac/institutes/grid.258041.a", 
              "name": [
                "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Stevens", 
            "givenName": "Scott P.", 
            "id": "sg:person.012120605565.18", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012120605565.18"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US", 
              "id": "http://www.grid.ac/institutes/grid.258041.a", 
              "name": [
                "Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Brookshire", 
            "givenName": "Robert G.", 
            "id": "sg:person.0614606364.33", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0614606364.33"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/bf01655906", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1000216607", 
              "https://doi.org/10.1007/bf01655906"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2001-04", 
        "datePublishedReg": "2001-04-01", 
        "description": "Recent interest in neural networks by researchers across a wide spectrum of disciplines has provided convincing evidence of their ability to address classification problems. In this article, we consider the issue of evaluating the predictive capability of neural networks when the output values are to be treated as probabilities. We propose the use of a variant of a chi-square statistic, based on the Hosmer\u2013Lemeshow statistic from logistic regression, to measure the goodness-of-fit of neural network models for two-group membership problems. Through experimentation with a large real-world database, we demonstrate the application of this statistic, and examine the effects of varying the number of nodes in the hidden layer on its value. Our empirical results suggest that this statistic can be very useful in identifying significant differences in the probability estimation accuracy of neural network models.", 
        "genre": "article", 
        "id": "sg:pub.10.1007/s005210170017", 
        "inLanguage": "en", 
        "isAccessibleForFree": false, 
        "isPartOf": [
          {
            "id": "sg:journal.1104357", 
            "issn": [
              "0941-0643", 
              "1433-3058"
            ], 
            "name": "Neural Computing and Applications", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "1", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "10"
          }
        ], 
        "keywords": [
          "neural network model", 
          "neural network", 
          "real-world databases", 
          "network model", 
          "large real-world databases", 
          "number of nodes", 
          "hidden layer", 
          "classification problem", 
          "empirical evaluation", 
          "estimation accuracy", 
          "probability estimation", 
          "network", 
          "membership problem", 
          "output values", 
          "empirical results", 
          "nodes", 
          "capability", 
          "predictive capability", 
          "database", 
          "experimentation", 
          "accuracy", 
          "applications", 
          "estimation", 
          "model", 
          "recent interest", 
          "chi-square statistics", 
          "statistics", 
          "researchers", 
          "issues", 
          "Hosmer-Lemeshow statistic", 
          "wide spectrum", 
          "problem", 
          "probability", 
          "disciplines", 
          "interest", 
          "evaluation", 
          "number", 
          "goodness", 
          "use", 
          "results", 
          "ability", 
          "variants", 
          "article", 
          "layer", 
          "regression", 
          "fit", 
          "values", 
          "logistic regression", 
          "spectra", 
          "differences", 
          "effect", 
          "evidence", 
          "convincing evidence", 
          "significant differences", 
          "two-group membership problems", 
          "probability estimation accuracy"
        ], 
        "name": "An Empirical Evaluation of Probability Estimation with Neural Networks", 
        "pagination": "48-55", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1018731983"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s005210170017"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s005210170017", 
          "https://app.dimensions.ai/details/publication/pub.1018731983"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2022-01-01T18:11", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20220101/entities/gbq_results/article/article_343.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1007/s005210170017"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s005210170017'


     

    This table displays all metadata directly associated to this object as RDF triples.

    132 TRIPLES      22 PREDICATES      82 URIs      73 LITERALS      6 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s005210170017 schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 schema:author Nd9ab5e3aed874503bf90ebd0e441f780
    4 schema:citation sg:pub.10.1007/bf01655906
    5 schema:datePublished 2001-04
    6 schema:datePublishedReg 2001-04-01
    7 schema:description Recent interest in neural networks by researchers across a wide spectrum of disciplines has provided convincing evidence of their ability to address classification problems. In this article, we consider the issue of evaluating the predictive capability of neural networks when the output values are to be treated as probabilities. We propose the use of a variant of a chi-square statistic, based on the Hosmer–Lemeshow statistic from logistic regression, to measure the goodness-of-fit of neural network models for two-group membership problems. Through experimentation with a large real-world database, we demonstrate the application of this statistic, and examine the effects of varying the number of nodes in the hidden layer on its value. Our empirical results suggest that this statistic can be very useful in identifying significant differences in the probability estimation accuracy of neural network models.
    8 schema:genre article
    9 schema:inLanguage en
    10 schema:isAccessibleForFree false
    11 schema:isPartOf N94605e46a3e74738b96d67a49c316b69
    12 Ndde42e7c1d7242b0a0cd12094efcc24b
    13 sg:journal.1104357
    14 schema:keywords Hosmer-Lemeshow statistic
    15 ability
    16 accuracy
    17 applications
    18 article
    19 capability
    20 chi-square statistics
    21 classification problem
    22 convincing evidence
    23 database
    24 differences
    25 disciplines
    26 effect
    27 empirical evaluation
    28 empirical results
    29 estimation
    30 estimation accuracy
    31 evaluation
    32 evidence
    33 experimentation
    34 fit
    35 goodness
    36 hidden layer
    37 interest
    38 issues
    39 large real-world databases
    40 layer
    41 logistic regression
    42 membership problem
    43 model
    44 network
    45 network model
    46 neural network
    47 neural network model
    48 nodes
    49 number
    50 number of nodes
    51 output values
    52 predictive capability
    53 probability
    54 probability estimation
    55 probability estimation accuracy
    56 problem
    57 real-world databases
    58 recent interest
    59 regression
    60 researchers
    61 results
    62 significant differences
    63 spectra
    64 statistics
    65 two-group membership problems
    66 use
    67 values
    68 variants
    69 wide spectrum
    70 schema:name An Empirical Evaluation of Probability Estimation with Neural Networks
    71 schema:pagination 48-55
    72 schema:productId N2f6d3887ac2d427ba66a6b6efc06ed2c
    73 N92fe2c329c0243e38a59331383473137
    74 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018731983
    75 https://doi.org/10.1007/s005210170017
    76 schema:sdDatePublished 2022-01-01T18:11
    77 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    78 schema:sdPublisher Nfaaeb45dff99434fb40cf707d93f4df3
    79 schema:url https://doi.org/10.1007/s005210170017
    80 sgo:license sg:explorer/license/
    81 sgo:sdDataset articles
    82 rdf:type schema:ScholarlyArticle
    83 N147d6350478642efab2c35e40c8cf59f rdf:first sg:person.0614606364.33
    84 rdf:rest rdf:nil
    85 N235c3f24c0da4f33ad61f5aca4a81102 rdf:first sg:person.012120605565.18
    86 rdf:rest N147d6350478642efab2c35e40c8cf59f
    87 N2f6d3887ac2d427ba66a6b6efc06ed2c schema:name doi
    88 schema:value 10.1007/s005210170017
    89 rdf:type schema:PropertyValue
    90 N92fe2c329c0243e38a59331383473137 schema:name dimensions_id
    91 schema:value pub.1018731983
    92 rdf:type schema:PropertyValue
    93 N94605e46a3e74738b96d67a49c316b69 schema:issueNumber 1
    94 rdf:type schema:PublicationIssue
    95 Nd9ab5e3aed874503bf90ebd0e441f780 rdf:first sg:person.012512771025.25
    96 rdf:rest N235c3f24c0da4f33ad61f5aca4a81102
    97 Ndde42e7c1d7242b0a0cd12094efcc24b schema:volumeNumber 10
    98 rdf:type schema:PublicationVolume
    99 Nfaaeb45dff99434fb40cf707d93f4df3 schema:name Springer Nature - SN SciGraph project
    100 rdf:type schema:Organization
    101 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    102 schema:name Information and Computing Sciences
    103 rdf:type schema:DefinedTerm
    104 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    105 schema:name Artificial Intelligence and Image Processing
    106 rdf:type schema:DefinedTerm
    107 sg:journal.1104357 schema:issn 0941-0643
    108 1433-3058
    109 schema:name Neural Computing and Applications
    110 schema:publisher Springer Nature
    111 rdf:type schema:Periodical
    112 sg:person.012120605565.18 schema:affiliation grid-institutes:grid.258041.a
    113 schema:familyName Stevens
    114 schema:givenName Scott P.
    115 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012120605565.18
    116 rdf:type schema:Person
    117 sg:person.012512771025.25 schema:affiliation grid-institutes:grid.258041.a
    118 schema:familyName Palocsay
    119 schema:givenName Susan W.
    120 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012512771025.25
    121 rdf:type schema:Person
    122 sg:person.0614606364.33 schema:affiliation grid-institutes:grid.258041.a
    123 schema:familyName Brookshire
    124 schema:givenName Robert G.
    125 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0614606364.33
    126 rdf:type schema:Person
    127 sg:pub.10.1007/bf01655906 schema:sameAs https://app.dimensions.ai/details/publication/pub.1000216607
    128 https://doi.org/10.1007/bf01655906
    129 rdf:type schema:CreativeWork
    130 grid-institutes:grid.258041.a schema:alternateName Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US
    131 schema:name Computer Information Systems/Operations Management Program, James Madison University, Harrisonburg, VA, USA, US
    132 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...