Deep Learning for Isotropic Super-Resolution from Non-isotropic 3D Electron Microscopy View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2017

AUTHORS

Larissa Heinrich , John A. Bogovic , Stephan Saalfeld

ABSTRACT

The most sophisticated existing methods to generate 3D isotropic super-resolution (SR) from non-isotropic electron microscopy (EM) are based on learned dictionaries. Unfortunately, none of the existing methods generate practically satisfying results. For 2D natural images, recently developed super-resolution methods that use deep learning have been shown to significantly outperform the previous state of the art. We have adapted one of the most successful architectures (FSRCNN) for 3D super-resolution, and compared its performance to a 3D U-Net architecture that has not been used previously to generate super-resolution. We trained both architectures on artificially downscaled isotropic ground truth from focused ion beam milling scanning EM (FIB-SEM) and tested the performance for various hyperparameter settings. Our results indicate that both architectures can successfully generate 3D isotropic super-resolution from non-isotropic EM, with the U-Net performing consistently better. We propose several promising directions for practical application. More... »

PAGES

135-143

References to SciGraph publications

  • 2016. Accelerating the Super-Resolution Convolutional Neural Network in COMPUTER VISION – ECCV 2016
  • 2016. Perceptual Losses for Real-Time Style Transfer and Super-Resolution in COMPUTER VISION – ECCV 2016
  • 2011. High Resolution Segmentation of Neuronal Tissues from Low Depth-Resolution EM Imagery in ENERGY MINIMIZATION METHODS IN COMPUTER VISION AND PATTERN RECOGNITION
  • 2014. Learning a Deep Convolutional Network for Image Super-Resolution in COMPUTER VISION – ECCV 2014
  • 2015-04. Ultrastructurally smooth thick partitioning and volume stitching for large-scale connectomics in NATURE METHODS
  • 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation in MEDICAL IMAGE COMPUTING AND COMPUTER-ASSISTED INTERVENTION – MICCAI 2015
  • Book

    TITLE

    Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017

    ISBN

    978-3-319-66184-1
    978-3-319-66185-8

    Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/978-3-319-66185-8_16

    DOI

    http://dx.doi.org/10.1007/978-3-319-66185-8_16

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1091429553


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0299", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Other Physical Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/02", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Physical Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "name": [
                "HHMI Janelia Research Campus"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Heinrich", 
            "givenName": "Larissa", 
            "id": "sg:person.010433417027.83", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010433417027.83"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "HHMI Janelia Research Campus"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Bogovic", 
            "givenName": "John A.", 
            "id": "sg:person.01143476206.54", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01143476206.54"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "HHMI Janelia Research Campus"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Saalfeld", 
            "givenName": "Stephan", 
            "id": "sg:person.0737142467.16", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0737142467.16"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/978-3-642-23094-3_19", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1001869705", 
              "https://doi.org/10.1007/978-3-642-23094-3_19"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-23094-3_19", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1001869705", 
              "https://doi.org/10.1007/978-3-642-23094-3_19"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.3389/fnana.2016.00062", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1005577304"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1038/nmeth.3292", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1007446959", 
              "https://doi.org/10.1038/nmeth.3292"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-46475-6_25", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1011486685", 
              "https://doi.org/10.1007/978-3-319-46475-6_25"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.conb.2014.01.019", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1014354610"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-24574-4_28", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1017774818", 
              "https://doi.org/10.1007/978-3-319-24574-4_28"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-46475-6_43", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1018034649", 
              "https://doi.org/10.1007/978-3-319-46475-6_43"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-10593-2_13", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1022837810", 
              "https://doi.org/10.1007/978-3-319-10593-2_13"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1093/bioinformatics/btw794", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1085373110"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/iccv.2009.5459271", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093345646"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2010.5539846", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093509757"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2017.19", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095836438"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2017", 
        "datePublishedReg": "2017-01-01", 
        "description": "The most sophisticated existing methods to generate 3D isotropic super-resolution (SR) from non-isotropic electron microscopy (EM) are based on learned dictionaries. Unfortunately, none of the existing methods generate practically satisfying results. For 2D natural images, recently developed super-resolution methods that use deep learning have been shown to significantly outperform the previous state of the art. We have adapted one of the most successful architectures (FSRCNN) for 3D super-resolution, and compared its performance to a 3D U-Net architecture that has not been used previously to generate super-resolution. We trained both architectures on artificially downscaled isotropic ground truth from focused ion beam milling scanning EM (FIB-SEM) and tested the performance for various hyperparameter settings. Our results indicate that both architectures can successfully generate 3D isotropic super-resolution from non-isotropic EM, with the U-Net performing consistently better. We propose several promising directions for practical application.", 
        "editor": [
          {
            "familyName": "Descoteaux", 
            "givenName": "Maxime", 
            "type": "Person"
          }, 
          {
            "familyName": "Maier-Hein", 
            "givenName": "Lena", 
            "type": "Person"
          }, 
          {
            "familyName": "Franz", 
            "givenName": "Alfred", 
            "type": "Person"
          }, 
          {
            "familyName": "Jannin", 
            "givenName": "Pierre", 
            "type": "Person"
          }, 
          {
            "familyName": "Collins", 
            "givenName": "D. Louis", 
            "type": "Person"
          }, 
          {
            "familyName": "Duchesne", 
            "givenName": "Simon", 
            "type": "Person"
          }
        ], 
        "genre": "chapter", 
        "id": "sg:pub.10.1007/978-3-319-66185-8_16", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": true, 
        "isPartOf": {
          "isbn": [
            "978-3-319-66184-1", 
            "978-3-319-66185-8"
          ], 
          "name": "Medical Image Computing and Computer-Assisted Intervention \u2212 MICCAI 2017", 
          "type": "Book"
        }, 
        "name": "Deep Learning for Isotropic Super-Resolution from Non-isotropic 3D Electron Microscopy", 
        "pagination": "135-143", 
        "productId": [
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/978-3-319-66185-8_16"
            ]
          }, 
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "c3b8786aa222fdfb06a41d49a1eaa1e7096c07d47c2d9fbd8554ff0fa3b4e2aa"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1091429553"
            ]
          }
        ], 
        "publisher": {
          "location": "Cham", 
          "name": "Springer International Publishing", 
          "type": "Organisation"
        }, 
        "sameAs": [
          "https://doi.org/10.1007/978-3-319-66185-8_16", 
          "https://app.dimensions.ai/details/publication/pub.1091429553"
        ], 
        "sdDataset": "chapters", 
        "sdDatePublished": "2019-04-16T00:31", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8697_00000601.jsonl", 
        "type": "Chapter", 
        "url": "http://link.springer.com/10.1007/978-3-319-66185-8_16"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-66185-8_16'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-66185-8_16'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-66185-8_16'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-66185-8_16'


     

    This table displays all metadata directly associated to this object as RDF triples.

    149 TRIPLES      23 PREDICATES      39 URIs      20 LITERALS      8 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/978-3-319-66185-8_16 schema:about anzsrc-for:02
    2 anzsrc-for:0299
    3 schema:author N82397582ee9441559f8f0309f893539f
    4 schema:citation sg:pub.10.1007/978-3-319-10593-2_13
    5 sg:pub.10.1007/978-3-319-24574-4_28
    6 sg:pub.10.1007/978-3-319-46475-6_25
    7 sg:pub.10.1007/978-3-319-46475-6_43
    8 sg:pub.10.1007/978-3-642-23094-3_19
    9 sg:pub.10.1038/nmeth.3292
    10 https://doi.org/10.1016/j.conb.2014.01.019
    11 https://doi.org/10.1093/bioinformatics/btw794
    12 https://doi.org/10.1109/cvpr.2010.5539846
    13 https://doi.org/10.1109/cvpr.2017.19
    14 https://doi.org/10.1109/iccv.2009.5459271
    15 https://doi.org/10.3389/fnana.2016.00062
    16 schema:datePublished 2017
    17 schema:datePublishedReg 2017-01-01
    18 schema:description The most sophisticated existing methods to generate 3D isotropic super-resolution (SR) from non-isotropic electron microscopy (EM) are based on learned dictionaries. Unfortunately, none of the existing methods generate practically satisfying results. For 2D natural images, recently developed super-resolution methods that use deep learning have been shown to significantly outperform the previous state of the art. We have adapted one of the most successful architectures (FSRCNN) for 3D super-resolution, and compared its performance to a 3D U-Net architecture that has not been used previously to generate super-resolution. We trained both architectures on artificially downscaled isotropic ground truth from focused ion beam milling scanning EM (FIB-SEM) and tested the performance for various hyperparameter settings. Our results indicate that both architectures can successfully generate 3D isotropic super-resolution from non-isotropic EM, with the U-Net performing consistently better. We propose several promising directions for practical application.
    19 schema:editor Ne02d865f369c43e5803409cd21fd9cd9
    20 schema:genre chapter
    21 schema:inLanguage en
    22 schema:isAccessibleForFree true
    23 schema:isPartOf N0edf227f327c4443be0b717a0ccf8236
    24 schema:name Deep Learning for Isotropic Super-Resolution from Non-isotropic 3D Electron Microscopy
    25 schema:pagination 135-143
    26 schema:productId N0c2a8fdb8b214b9886d1687be921da70
    27 N147b045904c24a0e99ff024e6316767b
    28 N36eee72dff45451480ee46967624c36c
    29 schema:publisher N11d9081902d74ca3ae76601b28bb76a8
    30 schema:sameAs https://app.dimensions.ai/details/publication/pub.1091429553
    31 https://doi.org/10.1007/978-3-319-66185-8_16
    32 schema:sdDatePublished 2019-04-16T00:31
    33 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    34 schema:sdPublisher N6f19a9b29d194fb3b68be8848357da98
    35 schema:url http://link.springer.com/10.1007/978-3-319-66185-8_16
    36 sgo:license sg:explorer/license/
    37 sgo:sdDataset chapters
    38 rdf:type schema:Chapter
    39 N036bc82d3e554db5bdad515cd77de073 schema:familyName Descoteaux
    40 schema:givenName Maxime
    41 rdf:type schema:Person
    42 N04260ec5fffb431ab728f7569cc50442 rdf:first N32a96e08dc4d4783a807fa7aec82b545
    43 rdf:rest N3793e6f5a5794d05a4b2333eaef434ac
    44 N0c2a8fdb8b214b9886d1687be921da70 schema:name dimensions_id
    45 schema:value pub.1091429553
    46 rdf:type schema:PropertyValue
    47 N0edf227f327c4443be0b717a0ccf8236 schema:isbn 978-3-319-66184-1
    48 978-3-319-66185-8
    49 schema:name Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017
    50 rdf:type schema:Book
    51 N11d9081902d74ca3ae76601b28bb76a8 schema:location Cham
    52 schema:name Springer International Publishing
    53 rdf:type schema:Organisation
    54 N147b045904c24a0e99ff024e6316767b schema:name doi
    55 schema:value 10.1007/978-3-319-66185-8_16
    56 rdf:type schema:PropertyValue
    57 N1afb99e4ee4242e398e8f5b5980ae937 schema:familyName Collins
    58 schema:givenName D. Louis
    59 rdf:type schema:Person
    60 N32a96e08dc4d4783a807fa7aec82b545 schema:familyName Franz
    61 schema:givenName Alfred
    62 rdf:type schema:Person
    63 N356a225486054b3485d9f9d601ac3720 schema:familyName Jannin
    64 schema:givenName Pierre
    65 rdf:type schema:Person
    66 N36eee72dff45451480ee46967624c36c schema:name readcube_id
    67 schema:value c3b8786aa222fdfb06a41d49a1eaa1e7096c07d47c2d9fbd8554ff0fa3b4e2aa
    68 rdf:type schema:PropertyValue
    69 N3793e6f5a5794d05a4b2333eaef434ac rdf:first N356a225486054b3485d9f9d601ac3720
    70 rdf:rest N4fffe639270944d1af1a018f948fb324
    71 N3a3847c78c6a452b9d590e5864528dc8 schema:familyName Duchesne
    72 schema:givenName Simon
    73 rdf:type schema:Person
    74 N4fffe639270944d1af1a018f948fb324 rdf:first N1afb99e4ee4242e398e8f5b5980ae937
    75 rdf:rest Naa6088c2da2f4c87980793dd594bcf5e
    76 N6ba28626262547d58782579896371835 rdf:first sg:person.0737142467.16
    77 rdf:rest rdf:nil
    78 N6f19a9b29d194fb3b68be8848357da98 schema:name Springer Nature - SN SciGraph project
    79 rdf:type schema:Organization
    80 N82397582ee9441559f8f0309f893539f rdf:first sg:person.010433417027.83
    81 rdf:rest Na4e5410028124849ab91c3574cf7ae4f
    82 Na3f3d0590a8b4af786b4c611a138dc0e schema:name HHMI Janelia Research Campus
    83 rdf:type schema:Organization
    84 Na4e5410028124849ab91c3574cf7ae4f rdf:first sg:person.01143476206.54
    85 rdf:rest N6ba28626262547d58782579896371835
    86 Naa6088c2da2f4c87980793dd594bcf5e rdf:first N3a3847c78c6a452b9d590e5864528dc8
    87 rdf:rest rdf:nil
    88 Nabecccbf6d11454782d33acce6feaff4 schema:familyName Maier-Hein
    89 schema:givenName Lena
    90 rdf:type schema:Person
    91 Nc7e85165588644cd840ab70b296009b3 schema:name HHMI Janelia Research Campus
    92 rdf:type schema:Organization
    93 Ne02d865f369c43e5803409cd21fd9cd9 rdf:first N036bc82d3e554db5bdad515cd77de073
    94 rdf:rest Nebafb4e5e6364909ac47aecbbba75088
    95 Nebafb4e5e6364909ac47aecbbba75088 rdf:first Nabecccbf6d11454782d33acce6feaff4
    96 rdf:rest N04260ec5fffb431ab728f7569cc50442
    97 Nffb9fb17ab8d4360a9b1617cb881e473 schema:name HHMI Janelia Research Campus
    98 rdf:type schema:Organization
    99 anzsrc-for:02 schema:inDefinedTermSet anzsrc-for:
    100 schema:name Physical Sciences
    101 rdf:type schema:DefinedTerm
    102 anzsrc-for:0299 schema:inDefinedTermSet anzsrc-for:
    103 schema:name Other Physical Sciences
    104 rdf:type schema:DefinedTerm
    105 sg:person.010433417027.83 schema:affiliation Nffb9fb17ab8d4360a9b1617cb881e473
    106 schema:familyName Heinrich
    107 schema:givenName Larissa
    108 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010433417027.83
    109 rdf:type schema:Person
    110 sg:person.01143476206.54 schema:affiliation Nc7e85165588644cd840ab70b296009b3
    111 schema:familyName Bogovic
    112 schema:givenName John A.
    113 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01143476206.54
    114 rdf:type schema:Person
    115 sg:person.0737142467.16 schema:affiliation Na3f3d0590a8b4af786b4c611a138dc0e
    116 schema:familyName Saalfeld
    117 schema:givenName Stephan
    118 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0737142467.16
    119 rdf:type schema:Person
    120 sg:pub.10.1007/978-3-319-10593-2_13 schema:sameAs https://app.dimensions.ai/details/publication/pub.1022837810
    121 https://doi.org/10.1007/978-3-319-10593-2_13
    122 rdf:type schema:CreativeWork
    123 sg:pub.10.1007/978-3-319-24574-4_28 schema:sameAs https://app.dimensions.ai/details/publication/pub.1017774818
    124 https://doi.org/10.1007/978-3-319-24574-4_28
    125 rdf:type schema:CreativeWork
    126 sg:pub.10.1007/978-3-319-46475-6_25 schema:sameAs https://app.dimensions.ai/details/publication/pub.1011486685
    127 https://doi.org/10.1007/978-3-319-46475-6_25
    128 rdf:type schema:CreativeWork
    129 sg:pub.10.1007/978-3-319-46475-6_43 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018034649
    130 https://doi.org/10.1007/978-3-319-46475-6_43
    131 rdf:type schema:CreativeWork
    132 sg:pub.10.1007/978-3-642-23094-3_19 schema:sameAs https://app.dimensions.ai/details/publication/pub.1001869705
    133 https://doi.org/10.1007/978-3-642-23094-3_19
    134 rdf:type schema:CreativeWork
    135 sg:pub.10.1038/nmeth.3292 schema:sameAs https://app.dimensions.ai/details/publication/pub.1007446959
    136 https://doi.org/10.1038/nmeth.3292
    137 rdf:type schema:CreativeWork
    138 https://doi.org/10.1016/j.conb.2014.01.019 schema:sameAs https://app.dimensions.ai/details/publication/pub.1014354610
    139 rdf:type schema:CreativeWork
    140 https://doi.org/10.1093/bioinformatics/btw794 schema:sameAs https://app.dimensions.ai/details/publication/pub.1085373110
    141 rdf:type schema:CreativeWork
    142 https://doi.org/10.1109/cvpr.2010.5539846 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093509757
    143 rdf:type schema:CreativeWork
    144 https://doi.org/10.1109/cvpr.2017.19 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095836438
    145 rdf:type schema:CreativeWork
    146 https://doi.org/10.1109/iccv.2009.5459271 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093345646
    147 rdf:type schema:CreativeWork
    148 https://doi.org/10.3389/fnana.2016.00062 schema:sameAs https://app.dimensions.ai/details/publication/pub.1005577304
    149 rdf:type schema:CreativeWork
     




    Preview window. Press ESC to close (or click here)


    ...