Efficient Approximations for the Marginal Likelihood of Bayesian Networks with Hidden Variables View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

1997-11

AUTHORS

David Maxwell Chickering, David Heckerman

ABSTRACT

We discuss Bayesian methods for model averaging and model selection among Bayesian-network models with hidden variables. In particular, we examine large-sample approximations for the marginal likelihood of naive-Bayes models in which the root node is hidden. Such models are useful for clustering or unsupervised learning. We consider a Laplace approximation and the less accurate but more computationally efficient approximation known as the Bayesian Information Criterion (BIC), which is equivalent to Rissanen's (1987) Minimum Description Length (MDL). Also, we consider approximations that ignore some off-diagonal elements of the observed information matrix and an approximation proposed by Cheeseman and Stutz (1995). We evaluate the accuracy of these approximations using a Monte-Carlo gold standard. In experiments with artificial and real examples, we find that (1) none of the approximations are accurate when used for model averaging, (2) all of the approximations, with the exception of BIC/MDL, are accurate for model selection, (3) among the accurate approximations, the Cheeseman–Stutz and Diagonal approximations are the most computationally efficient, (4) all of the approximations, with the exception of BIC/MDL, can be sensitive to the prior distribution over model parameters, and (5) the Cheeseman–Stutz approximation can be more accurate than the other approximations, including the Laplace approximation, in situations where the parameters in the maximum a posteriori configuration are near a boundary. More... »

PAGES

181-212

Journal

TITLE

Machine Learning

ISSUE

2-3

VOLUME

29

Author Affiliations

Identifiers

URI

http://scigraph.springernature.com/pub.10.1023/a:1007469629108

DOI

http://dx.doi.org/10.1023/a:1007469629108

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1044802083


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0104", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Statistics", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/01", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Mathematical Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Microsoft (United States)", 
          "id": "https://www.grid.ac/institutes/grid.419815.0", 
          "name": [
            "Microsoft Research, 98052-6399, Redmond, WA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Maxwell Chickering", 
        "givenName": "David", 
        "id": "sg:person.011240332636.47", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011240332636.47"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Microsoft (United States)", 
          "id": "https://www.grid.ac/institutes/grid.419815.0", 
          "name": [
            "Microsoft Research, 98052-6399, Redmond, WA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Heckerman", 
        "givenName": "David", 
        "id": "sg:person.01134362461.98", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01134362461.98"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.1162/neco.1992.4.3.448", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1005836706"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1162/neco.1992.4.3.415", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1016555682"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/b978-1-55860-332-5.50009-2", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1020634759"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-1-4899-1292-3_6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1022719367", 
          "https://doi.org/10.1007/978-1-4899-1292-3_6"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0031-3203(91)90074-f", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1030619715"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0031-3203(91)90074-f", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1030619715"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf00994016", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1035524560", 
          "https://doi.org/10.1007/bf00994016"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1214/aos/1176344136", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1044872629"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf00994110", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1046316965", 
          "https://doi.org/10.1007/bf00994110"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/b978-0-934613-41-5.50005-2", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1046930152"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1080/01621459.1991.10475130", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1058304159"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1080/01621459.1995.10476572", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1058304855"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1080/01621459.1995.10476592", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1058304875"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1080/01621459.1995.10476635", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1058304918"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1093/biomet/63.3.581", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1059418581"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/72.286919", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061218456"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/tpami.1984.4767596", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061742090"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1214/aos/1176350709", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1064409239"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1214/ss/1177010888", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1064409646"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.2307/1403615", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1069473898"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.2307/271063", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1070072554"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1613/jair.251", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1105538415"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1613/jair.62", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1105538472"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "1997-11", 
    "datePublishedReg": "1997-11-01", 
    "description": "We discuss Bayesian methods for model averaging and model selection among Bayesian-network models with hidden variables. In particular, we examine large-sample approximations for the marginal likelihood of naive-Bayes models in which the root node is hidden. Such models are useful for clustering or unsupervised learning. We consider a Laplace approximation and the less accurate but more computationally efficient approximation known as the Bayesian Information Criterion (BIC), which is equivalent to Rissanen's (1987) Minimum Description Length (MDL). Also, we consider approximations that ignore some off-diagonal elements of the observed information matrix and an approximation proposed by Cheeseman and Stutz (1995). We evaluate the accuracy of these approximations using a Monte-Carlo gold standard. In experiments with artificial and real examples, we find that (1) none of the approximations are accurate when used for model averaging, (2) all of the approximations, with the exception of BIC/MDL, are accurate for model selection, (3) among the accurate approximations, the Cheeseman\u2013Stutz and Diagonal approximations are the most computationally efficient, (4) all of the approximations, with the exception of BIC/MDL, can be sensitive to the prior distribution over model parameters, and (5) the Cheeseman\u2013Stutz approximation can be more accurate than the other approximations, including the Laplace approximation, in situations where the parameters in the maximum a posteriori configuration are near a boundary.", 
    "genre": "research_article", 
    "id": "sg:pub.10.1023/a:1007469629108", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": true, 
    "isPartOf": [
      {
        "id": "sg:journal.1125588", 
        "issn": [
          "0885-6125", 
          "1573-0565"
        ], 
        "name": "Machine Learning", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "2-3", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "29"
      }
    ], 
    "name": "Efficient Approximations for the Marginal Likelihood of Bayesian Networks with Hidden Variables", 
    "pagination": "181-212", 
    "productId": [
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "d568437a79020f0c2196ce254023d5be60467d962f8e92109acec53bfa6de076"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1023/a:1007469629108"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1044802083"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1023/a:1007469629108", 
      "https://app.dimensions.ai/details/publication/pub.1044802083"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2019-04-10T22:29", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8690_00000501.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "http://link.springer.com/10.1023/A:1007469629108"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1023/a:1007469629108'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1023/a:1007469629108'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1023/a:1007469629108'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1023/a:1007469629108'


 

This table displays all metadata directly associated to this object as RDF triples.

137 TRIPLES      21 PREDICATES      49 URIs      19 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1023/a:1007469629108 schema:about anzsrc-for:01
2 anzsrc-for:0104
3 schema:author Nf3ee5603613446488dc71312235bf346
4 schema:citation sg:pub.10.1007/978-1-4899-1292-3_6
5 sg:pub.10.1007/bf00994016
6 sg:pub.10.1007/bf00994110
7 https://doi.org/10.1016/0031-3203(91)90074-f
8 https://doi.org/10.1016/b978-0-934613-41-5.50005-2
9 https://doi.org/10.1016/b978-1-55860-332-5.50009-2
10 https://doi.org/10.1080/01621459.1991.10475130
11 https://doi.org/10.1080/01621459.1995.10476572
12 https://doi.org/10.1080/01621459.1995.10476592
13 https://doi.org/10.1080/01621459.1995.10476635
14 https://doi.org/10.1093/biomet/63.3.581
15 https://doi.org/10.1109/72.286919
16 https://doi.org/10.1109/tpami.1984.4767596
17 https://doi.org/10.1162/neco.1992.4.3.415
18 https://doi.org/10.1162/neco.1992.4.3.448
19 https://doi.org/10.1214/aos/1176344136
20 https://doi.org/10.1214/aos/1176350709
21 https://doi.org/10.1214/ss/1177010888
22 https://doi.org/10.1613/jair.251
23 https://doi.org/10.1613/jair.62
24 https://doi.org/10.2307/1403615
25 https://doi.org/10.2307/271063
26 schema:datePublished 1997-11
27 schema:datePublishedReg 1997-11-01
28 schema:description We discuss Bayesian methods for model averaging and model selection among Bayesian-network models with hidden variables. In particular, we examine large-sample approximations for the marginal likelihood of naive-Bayes models in which the root node is hidden. Such models are useful for clustering or unsupervised learning. We consider a Laplace approximation and the less accurate but more computationally efficient approximation known as the Bayesian Information Criterion (BIC), which is equivalent to Rissanen's (1987) Minimum Description Length (MDL). Also, we consider approximations that ignore some off-diagonal elements of the observed information matrix and an approximation proposed by Cheeseman and Stutz (1995). We evaluate the accuracy of these approximations using a Monte-Carlo gold standard. In experiments with artificial and real examples, we find that (1) none of the approximations are accurate when used for model averaging, (2) all of the approximations, with the exception of BIC/MDL, are accurate for model selection, (3) among the accurate approximations, the Cheeseman–Stutz and Diagonal approximations are the most computationally efficient, (4) all of the approximations, with the exception of BIC/MDL, can be sensitive to the prior distribution over model parameters, and (5) the Cheeseman–Stutz approximation can be more accurate than the other approximations, including the Laplace approximation, in situations where the parameters in the maximum a posteriori configuration are near a boundary.
29 schema:genre research_article
30 schema:inLanguage en
31 schema:isAccessibleForFree true
32 schema:isPartOf N1b682069001f4c35a2765b1f378844c9
33 Nfc2cba88dc5d47cab8a88da2e6cdbfa9
34 sg:journal.1125588
35 schema:name Efficient Approximations for the Marginal Likelihood of Bayesian Networks with Hidden Variables
36 schema:pagination 181-212
37 schema:productId N0f1e7310d4944389a4dcdabb36b3f51b
38 N1c1f2d7e77324cc3b7c8d72c92a72aa1
39 N79605f1f736b4c72b0a45c8e1818dcf3
40 schema:sameAs https://app.dimensions.ai/details/publication/pub.1044802083
41 https://doi.org/10.1023/a:1007469629108
42 schema:sdDatePublished 2019-04-10T22:29
43 schema:sdLicense https://scigraph.springernature.com/explorer/license/
44 schema:sdPublisher Neee0b077ad71449085f45e59d86b0a96
45 schema:url http://link.springer.com/10.1023/A:1007469629108
46 sgo:license sg:explorer/license/
47 sgo:sdDataset articles
48 rdf:type schema:ScholarlyArticle
49 N0f1e7310d4944389a4dcdabb36b3f51b schema:name readcube_id
50 schema:value d568437a79020f0c2196ce254023d5be60467d962f8e92109acec53bfa6de076
51 rdf:type schema:PropertyValue
52 N1b682069001f4c35a2765b1f378844c9 schema:volumeNumber 29
53 rdf:type schema:PublicationVolume
54 N1c1f2d7e77324cc3b7c8d72c92a72aa1 schema:name dimensions_id
55 schema:value pub.1044802083
56 rdf:type schema:PropertyValue
57 N582a1cb042ec4a5d88148b52e0b22eae rdf:first sg:person.01134362461.98
58 rdf:rest rdf:nil
59 N79605f1f736b4c72b0a45c8e1818dcf3 schema:name doi
60 schema:value 10.1023/a:1007469629108
61 rdf:type schema:PropertyValue
62 Neee0b077ad71449085f45e59d86b0a96 schema:name Springer Nature - SN SciGraph project
63 rdf:type schema:Organization
64 Nf3ee5603613446488dc71312235bf346 rdf:first sg:person.011240332636.47
65 rdf:rest N582a1cb042ec4a5d88148b52e0b22eae
66 Nfc2cba88dc5d47cab8a88da2e6cdbfa9 schema:issueNumber 2-3
67 rdf:type schema:PublicationIssue
68 anzsrc-for:01 schema:inDefinedTermSet anzsrc-for:
69 schema:name Mathematical Sciences
70 rdf:type schema:DefinedTerm
71 anzsrc-for:0104 schema:inDefinedTermSet anzsrc-for:
72 schema:name Statistics
73 rdf:type schema:DefinedTerm
74 sg:journal.1125588 schema:issn 0885-6125
75 1573-0565
76 schema:name Machine Learning
77 rdf:type schema:Periodical
78 sg:person.011240332636.47 schema:affiliation https://www.grid.ac/institutes/grid.419815.0
79 schema:familyName Maxwell Chickering
80 schema:givenName David
81 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011240332636.47
82 rdf:type schema:Person
83 sg:person.01134362461.98 schema:affiliation https://www.grid.ac/institutes/grid.419815.0
84 schema:familyName Heckerman
85 schema:givenName David
86 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01134362461.98
87 rdf:type schema:Person
88 sg:pub.10.1007/978-1-4899-1292-3_6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1022719367
89 https://doi.org/10.1007/978-1-4899-1292-3_6
90 rdf:type schema:CreativeWork
91 sg:pub.10.1007/bf00994016 schema:sameAs https://app.dimensions.ai/details/publication/pub.1035524560
92 https://doi.org/10.1007/bf00994016
93 rdf:type schema:CreativeWork
94 sg:pub.10.1007/bf00994110 schema:sameAs https://app.dimensions.ai/details/publication/pub.1046316965
95 https://doi.org/10.1007/bf00994110
96 rdf:type schema:CreativeWork
97 https://doi.org/10.1016/0031-3203(91)90074-f schema:sameAs https://app.dimensions.ai/details/publication/pub.1030619715
98 rdf:type schema:CreativeWork
99 https://doi.org/10.1016/b978-0-934613-41-5.50005-2 schema:sameAs https://app.dimensions.ai/details/publication/pub.1046930152
100 rdf:type schema:CreativeWork
101 https://doi.org/10.1016/b978-1-55860-332-5.50009-2 schema:sameAs https://app.dimensions.ai/details/publication/pub.1020634759
102 rdf:type schema:CreativeWork
103 https://doi.org/10.1080/01621459.1991.10475130 schema:sameAs https://app.dimensions.ai/details/publication/pub.1058304159
104 rdf:type schema:CreativeWork
105 https://doi.org/10.1080/01621459.1995.10476572 schema:sameAs https://app.dimensions.ai/details/publication/pub.1058304855
106 rdf:type schema:CreativeWork
107 https://doi.org/10.1080/01621459.1995.10476592 schema:sameAs https://app.dimensions.ai/details/publication/pub.1058304875
108 rdf:type schema:CreativeWork
109 https://doi.org/10.1080/01621459.1995.10476635 schema:sameAs https://app.dimensions.ai/details/publication/pub.1058304918
110 rdf:type schema:CreativeWork
111 https://doi.org/10.1093/biomet/63.3.581 schema:sameAs https://app.dimensions.ai/details/publication/pub.1059418581
112 rdf:type schema:CreativeWork
113 https://doi.org/10.1109/72.286919 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061218456
114 rdf:type schema:CreativeWork
115 https://doi.org/10.1109/tpami.1984.4767596 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061742090
116 rdf:type schema:CreativeWork
117 https://doi.org/10.1162/neco.1992.4.3.415 schema:sameAs https://app.dimensions.ai/details/publication/pub.1016555682
118 rdf:type schema:CreativeWork
119 https://doi.org/10.1162/neco.1992.4.3.448 schema:sameAs https://app.dimensions.ai/details/publication/pub.1005836706
120 rdf:type schema:CreativeWork
121 https://doi.org/10.1214/aos/1176344136 schema:sameAs https://app.dimensions.ai/details/publication/pub.1044872629
122 rdf:type schema:CreativeWork
123 https://doi.org/10.1214/aos/1176350709 schema:sameAs https://app.dimensions.ai/details/publication/pub.1064409239
124 rdf:type schema:CreativeWork
125 https://doi.org/10.1214/ss/1177010888 schema:sameAs https://app.dimensions.ai/details/publication/pub.1064409646
126 rdf:type schema:CreativeWork
127 https://doi.org/10.1613/jair.251 schema:sameAs https://app.dimensions.ai/details/publication/pub.1105538415
128 rdf:type schema:CreativeWork
129 https://doi.org/10.1613/jair.62 schema:sameAs https://app.dimensions.ai/details/publication/pub.1105538472
130 rdf:type schema:CreativeWork
131 https://doi.org/10.2307/1403615 schema:sameAs https://app.dimensions.ai/details/publication/pub.1069473898
132 rdf:type schema:CreativeWork
133 https://doi.org/10.2307/271063 schema:sameAs https://app.dimensions.ai/details/publication/pub.1070072554
134 rdf:type schema:CreativeWork
135 https://www.grid.ac/institutes/grid.419815.0 schema:alternateName Microsoft (United States)
136 schema:name Microsoft Research, 98052-6399, Redmond, WA
137 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...