Putting the User in the Loop for Image-Based Modeling View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

2014-05

AUTHORS

Adarsh Kowdle, Yao-Jen Chang, Andrew Gallagher, Dhruv Batra, Tsuhan Chen

ABSTRACT

We refer to the task of recovering the 3D structure of an object or a scene using 2D images as image-based modeling. In this paper, we formulate the task of recovering the 3D structure as a discrete optimization problem solved via energy minimization. In this standard framework of a Markov random field (MRF) defined over the image we present algorithms that allow the user to intuitively interact with the algorithm. We introduce an algorithm where the user guides the process of image-based modeling to find and model the object of interest by manually interacting with the nodes of the graph. We develop end user applications using this algorithm that allow object of interest 3D modeling on a mobile device and 3D printing of the object of interest. We also propose an alternate active learning algorithm that guides the user input. An initial attempt is made at reconstructing the scene without supervision. Given the reconstruction, an active learning algorithm uses intuitive cues to quantify the uncertainty of the algorithm and suggest regions, querying the user to provide support for the uncertain regions via simple scribbles. These constraints are used to update the unary and the pairwise energies that, when solved, lead to better reconstructions. We show through machine experiments and a user study that the proposed approach intelligently queries the users for constraints, and users achieve better reconstructions of the scene faster, especially for scenes with textureless surfaces lacking strong textural or structural cues that algorithms typically require. More... »

PAGES

30-48

References to SciGraph publications

  • 2004-09. Efficient Graph-Based Image Segmentation in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2004-09. Visual Modeling with a Hand-Held Camera in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2003-04. Relevance feedback in image retrieval: A comprehensive review in MULTIMEDIA SYSTEMS
  • 2012. iModel: Interactive Co-segmentation for Object of Interest 3D Modeling in TRENDS AND TOPICS IN COMPUTER VISION
  • 2007-10. Recovering Surface Layout from an Image in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2008. Towards Scalable Dataset Construction: An Active Learning Approach in COMPUTER VISION – ECCV 2008
  • 2007. Identifying Foreground from Multiple Images in COMPUTER VISION – ACCV 2007
  • 2012-12. User-Centric Learning and Evaluation of Interactive Segmentation Systems in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2011-07. Interactively Co-segmentating Topically Related Images with Intelligent Scribble Guidance in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2012. Multiple View Object Cosegmentation Using Appearance and Stereo Cues in COMPUTER VISION – ECCV 2012
  • 2006. Shape-from-Silhouette with Two Mirrors and an Uncalibrated Camera in COMPUTER VISION – ECCV 2006
  • 2002-04. A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2010-08. Multi-view Superpixel Stereo in Urban Environments in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2008-07. Detailed Real-Time Urban 3D Reconstruction from Video in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s11263-014-0704-x

    DOI

    http://dx.doi.org/10.1007/s11263-014-0704-x

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1005061829


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Cornell University", 
              "id": "https://www.grid.ac/institutes/grid.5386.8", 
              "name": [
                "Cornell University, Ithaca, NY, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Kowdle", 
            "givenName": "Adarsh", 
            "id": "sg:person.010315021706.95", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010315021706.95"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Siemens (United States)", 
              "id": "https://www.grid.ac/institutes/grid.419233.e", 
              "name": [
                "Siemens Corporation, Corporate Technology, Princeton, NJ, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Chang", 
            "givenName": "Yao-Jen", 
            "id": "sg:person.01345446770.94", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01345446770.94"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Cornell University", 
              "id": "https://www.grid.ac/institutes/grid.5386.8", 
              "name": [
                "Cornell University, Ithaca, NY, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Gallagher", 
            "givenName": "Andrew", 
            "id": "sg:person.07707707055.43", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07707707055.43"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Virginia Tech", 
              "id": "https://www.grid.ac/institutes/grid.438526.e", 
              "name": [
                "Virginia Tech, Blacksburg, VA, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Batra", 
            "givenName": "Dhruv", 
            "id": "sg:person.016177260105.00", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016177260105.00"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Cornell University", 
              "id": "https://www.grid.ac/institutes/grid.5386.8", 
              "name": [
                "Cornell University, Ithaca, NY, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Chen", 
            "givenName": "Tsuhan", 
            "id": "sg:person.012245072625.31", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012245072625.31"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1023/a:1014573219977", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1004426816", 
              "https://doi.org/10.1023/a:1014573219977"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.cviu.2008.07.002", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1008932526"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1023/b:visi.0000022288.19776.77", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1009092998", 
              "https://doi.org/10.1023/b:visi.0000022288.19776.77"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.cviu.2006.07.011", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1013868408"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-35740-4_17", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1014018594", 
              "https://doi.org/10.1007/978-3-642-35740-4_17"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-007-0086-4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1015440148", 
              "https://doi.org/10.1007/s11263-007-0086-4"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-010-0327-9", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1016062726", 
              "https://doi.org/10.1007/s11263-010-0327-9"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-010-0327-9", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1016062726", 
              "https://doi.org/10.1007/s11263-010-0327-9"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-012-0537-4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1018298508", 
              "https://doi.org/10.1007/s11263-012-0537-4"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s00530-002-0070-3", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1026592439", 
              "https://doi.org/10.1007/s00530-002-0070-3"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/s0167-8655(02)00370-7", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1026695583"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1457515.1409112", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1029206453"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/11744047_13", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1031783594", 
              "https://doi.org/10.1007/11744047_13"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/11744047_13", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1031783594", 
              "https://doi.org/10.1007/11744047_13"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1023/b:visi.0000025798.50602.3a", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1036329061", 
              "https://doi.org/10.1023/b:visi.0000025798.50602.3a"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-540-76390-1_57", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1037301069", 
              "https://doi.org/10.1007/978-3-540-76390-1_57"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-540-76390-1_57", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1037301069", 
              "https://doi.org/10.1007/978-3-540-76390-1_57"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.cviu.2011.02.011", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1041995597"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-006-0031-y", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1043247765", 
              "https://doi.org/10.1007/s11263-006-0031-y"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/237170.237191", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1045338552"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-540-88682-2_8", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1045945012", 
              "https://doi.org/10.1007/978-3-540-88682-2_8"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1073204.1073232", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1051131415"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-33715-4_57", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1051335619", 
              "https://doi.org/10.1007/978-3-642-33715-4_57"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-010-0415-x", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1051801565", 
              "https://doi.org/10.1007/s11263-010-0415-x"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1179352.1141964", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1052911580"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1006/cviu.1993.1030", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1054487278"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/34.1000236", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061155588"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/34.969114", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061157335"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tip.2008.924286", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061642123"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tpami.2004.1262177", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061742646"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tpami.2004.60", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061742742"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tpami.2008.132", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061743494"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tpami.2009.161", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061743739"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1141911.1141964", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1063151975"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1276377.1276485", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1063153865"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1409060.1409112", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1063155288"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2011.5995530", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093473548"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/iccv.2003.1238391", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093735574"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2010.5539804", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093859913"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icpr.2008.4761602", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093884142"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2010.5540193", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094320167"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/iccv.2007.4408844", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094384110"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/iccv.1999.791253", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094521783"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2011.5995638", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094630828"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2010.5539802", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094846270"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/iccv.2009.5459417", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095016346"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/iccv.2009.5459145", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095132772"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2010.5540055", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095170315"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icip.2011.6116190", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095189133"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cvpr.2009.5206651", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095223629"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/iccv.2007.4408933", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095774143"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.5244/c.21.58", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1099341553"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2014-05", 
        "datePublishedReg": "2014-05-01", 
        "description": "We refer to the task of recovering the 3D structure of an object or a scene using 2D images as image-based modeling. In this paper, we formulate the task of recovering the 3D structure as a discrete optimization problem solved via energy minimization. In this standard framework of a Markov random field (MRF) defined over the image we present algorithms that allow the user to intuitively interact with the algorithm. We introduce an algorithm where the user guides the process of image-based modeling to find and model the object of interest by manually interacting with the nodes of the graph. We develop end user applications using this algorithm that allow object of interest 3D modeling on a mobile device and 3D printing of the object of interest. We also propose an alternate active learning algorithm that guides the user input. An initial attempt is made at reconstructing the scene without supervision. Given the reconstruction, an active learning algorithm uses intuitive cues to quantify the uncertainty of the algorithm and suggest regions, querying the user to provide support for the uncertain regions via simple scribbles. These constraints are used to update the unary and the pairwise energies that, when solved, lead to better reconstructions. We show through machine experiments and a user study that the proposed approach intelligently queries the users for constraints, and users achieve better reconstructions of the scene faster, especially for scenes with textureless surfaces lacking strong textural or structural cues that algorithms typically require.", 
        "genre": "research_article", 
        "id": "sg:pub.10.1007/s11263-014-0704-x", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": true, 
        "isPartOf": [
          {
            "id": "sg:journal.1032807", 
            "issn": [
              "0920-5691", 
              "1573-1405"
            ], 
            "name": "International Journal of Computer Vision", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "1-2", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "108"
          }
        ], 
        "name": "Putting the User in the Loop for Image-Based Modeling", 
        "pagination": "30-48", 
        "productId": [
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "181dff8a70dddf68aa8c066786c536a84c2c7099bd96e77758d5634253965cdd"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s11263-014-0704-x"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1005061829"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s11263-014-0704-x", 
          "https://app.dimensions.ai/details/publication/pub.1005061829"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2019-04-10T20:48", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8684_00000520.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "http://link.springer.com/10.1007%2Fs11263-014-0704-x"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s11263-014-0704-x'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s11263-014-0704-x'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s11263-014-0704-x'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s11263-014-0704-x'


     

    This table displays all metadata directly associated to this object as RDF triples.

    256 TRIPLES      21 PREDICATES      76 URIs      19 LITERALS      7 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s11263-014-0704-x schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 schema:author N43dd90f35b1f4410abf717be6645dc4b
    4 schema:citation sg:pub.10.1007/11744047_13
    5 sg:pub.10.1007/978-3-540-76390-1_57
    6 sg:pub.10.1007/978-3-540-88682-2_8
    7 sg:pub.10.1007/978-3-642-33715-4_57
    8 sg:pub.10.1007/978-3-642-35740-4_17
    9 sg:pub.10.1007/s00530-002-0070-3
    10 sg:pub.10.1007/s11263-006-0031-y
    11 sg:pub.10.1007/s11263-007-0086-4
    12 sg:pub.10.1007/s11263-010-0327-9
    13 sg:pub.10.1007/s11263-010-0415-x
    14 sg:pub.10.1007/s11263-012-0537-4
    15 sg:pub.10.1023/a:1014573219977
    16 sg:pub.10.1023/b:visi.0000022288.19776.77
    17 sg:pub.10.1023/b:visi.0000025798.50602.3a
    18 https://doi.org/10.1006/cviu.1993.1030
    19 https://doi.org/10.1016/j.cviu.2006.07.011
    20 https://doi.org/10.1016/j.cviu.2008.07.002
    21 https://doi.org/10.1016/j.cviu.2011.02.011
    22 https://doi.org/10.1016/s0167-8655(02)00370-7
    23 https://doi.org/10.1109/34.1000236
    24 https://doi.org/10.1109/34.969114
    25 https://doi.org/10.1109/cvpr.2009.5206651
    26 https://doi.org/10.1109/cvpr.2010.5539802
    27 https://doi.org/10.1109/cvpr.2010.5539804
    28 https://doi.org/10.1109/cvpr.2010.5540055
    29 https://doi.org/10.1109/cvpr.2010.5540193
    30 https://doi.org/10.1109/cvpr.2011.5995530
    31 https://doi.org/10.1109/cvpr.2011.5995638
    32 https://doi.org/10.1109/iccv.1999.791253
    33 https://doi.org/10.1109/iccv.2003.1238391
    34 https://doi.org/10.1109/iccv.2007.4408844
    35 https://doi.org/10.1109/iccv.2007.4408933
    36 https://doi.org/10.1109/iccv.2009.5459145
    37 https://doi.org/10.1109/iccv.2009.5459417
    38 https://doi.org/10.1109/icip.2011.6116190
    39 https://doi.org/10.1109/icpr.2008.4761602
    40 https://doi.org/10.1109/tip.2008.924286
    41 https://doi.org/10.1109/tpami.2004.1262177
    42 https://doi.org/10.1109/tpami.2004.60
    43 https://doi.org/10.1109/tpami.2008.132
    44 https://doi.org/10.1109/tpami.2009.161
    45 https://doi.org/10.1145/1073204.1073232
    46 https://doi.org/10.1145/1141911.1141964
    47 https://doi.org/10.1145/1179352.1141964
    48 https://doi.org/10.1145/1276377.1276485
    49 https://doi.org/10.1145/1409060.1409112
    50 https://doi.org/10.1145/1457515.1409112
    51 https://doi.org/10.1145/237170.237191
    52 https://doi.org/10.5244/c.21.58
    53 schema:datePublished 2014-05
    54 schema:datePublishedReg 2014-05-01
    55 schema:description We refer to the task of recovering the 3D structure of an object or a scene using 2D images as image-based modeling. In this paper, we formulate the task of recovering the 3D structure as a discrete optimization problem solved via energy minimization. In this standard framework of a Markov random field (MRF) defined over the image we present algorithms that allow the user to intuitively interact with the algorithm. We introduce an algorithm where the user guides the process of image-based modeling to find and model the object of interest by manually interacting with the nodes of the graph. We develop end user applications using this algorithm that allow object of interest 3D modeling on a mobile device and 3D printing of the object of interest. We also propose an alternate active learning algorithm that guides the user input. An initial attempt is made at reconstructing the scene without supervision. Given the reconstruction, an active learning algorithm uses intuitive cues to quantify the uncertainty of the algorithm and suggest regions, querying the user to provide support for the uncertain regions via simple scribbles. These constraints are used to update the unary and the pairwise energies that, when solved, lead to better reconstructions. We show through machine experiments and a user study that the proposed approach intelligently queries the users for constraints, and users achieve better reconstructions of the scene faster, especially for scenes with textureless surfaces lacking strong textural or structural cues that algorithms typically require.
    56 schema:genre research_article
    57 schema:inLanguage en
    58 schema:isAccessibleForFree true
    59 schema:isPartOf Ne4f83d06fa884e7184e256552fe97154
    60 Nf10cb92630c74fd0a2ee5682e51614e0
    61 sg:journal.1032807
    62 schema:name Putting the User in the Loop for Image-Based Modeling
    63 schema:pagination 30-48
    64 schema:productId N027f1e3707044f9292a2f2f02dc170b1
    65 N11567011a1d0433d8f6e585e3147e6fb
    66 N93fae6223f694cf394f004ad18da9cb6
    67 schema:sameAs https://app.dimensions.ai/details/publication/pub.1005061829
    68 https://doi.org/10.1007/s11263-014-0704-x
    69 schema:sdDatePublished 2019-04-10T20:48
    70 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    71 schema:sdPublisher N84196c728bab427a8a643541a8775683
    72 schema:url http://link.springer.com/10.1007%2Fs11263-014-0704-x
    73 sgo:license sg:explorer/license/
    74 sgo:sdDataset articles
    75 rdf:type schema:ScholarlyArticle
    76 N027f1e3707044f9292a2f2f02dc170b1 schema:name doi
    77 schema:value 10.1007/s11263-014-0704-x
    78 rdf:type schema:PropertyValue
    79 N11567011a1d0433d8f6e585e3147e6fb schema:name readcube_id
    80 schema:value 181dff8a70dddf68aa8c066786c536a84c2c7099bd96e77758d5634253965cdd
    81 rdf:type schema:PropertyValue
    82 N1e1d4560237846d8977907a92acb7793 rdf:first sg:person.016177260105.00
    83 rdf:rest N8c698e4df2a143b1a31d9a5d14ce7bff
    84 N43dd90f35b1f4410abf717be6645dc4b rdf:first sg:person.010315021706.95
    85 rdf:rest N782e00d0113f4b2fa47698052539122c
    86 N6cfebf62e9074c3994b5987853a56242 rdf:first sg:person.07707707055.43
    87 rdf:rest N1e1d4560237846d8977907a92acb7793
    88 N782e00d0113f4b2fa47698052539122c rdf:first sg:person.01345446770.94
    89 rdf:rest N6cfebf62e9074c3994b5987853a56242
    90 N84196c728bab427a8a643541a8775683 schema:name Springer Nature - SN SciGraph project
    91 rdf:type schema:Organization
    92 N8c698e4df2a143b1a31d9a5d14ce7bff rdf:first sg:person.012245072625.31
    93 rdf:rest rdf:nil
    94 N93fae6223f694cf394f004ad18da9cb6 schema:name dimensions_id
    95 schema:value pub.1005061829
    96 rdf:type schema:PropertyValue
    97 Ne4f83d06fa884e7184e256552fe97154 schema:volumeNumber 108
    98 rdf:type schema:PublicationVolume
    99 Nf10cb92630c74fd0a2ee5682e51614e0 schema:issueNumber 1-2
    100 rdf:type schema:PublicationIssue
    101 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    102 schema:name Information and Computing Sciences
    103 rdf:type schema:DefinedTerm
    104 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    105 schema:name Artificial Intelligence and Image Processing
    106 rdf:type schema:DefinedTerm
    107 sg:journal.1032807 schema:issn 0920-5691
    108 1573-1405
    109 schema:name International Journal of Computer Vision
    110 rdf:type schema:Periodical
    111 sg:person.010315021706.95 schema:affiliation https://www.grid.ac/institutes/grid.5386.8
    112 schema:familyName Kowdle
    113 schema:givenName Adarsh
    114 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010315021706.95
    115 rdf:type schema:Person
    116 sg:person.012245072625.31 schema:affiliation https://www.grid.ac/institutes/grid.5386.8
    117 schema:familyName Chen
    118 schema:givenName Tsuhan
    119 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012245072625.31
    120 rdf:type schema:Person
    121 sg:person.01345446770.94 schema:affiliation https://www.grid.ac/institutes/grid.419233.e
    122 schema:familyName Chang
    123 schema:givenName Yao-Jen
    124 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01345446770.94
    125 rdf:type schema:Person
    126 sg:person.016177260105.00 schema:affiliation https://www.grid.ac/institutes/grid.438526.e
    127 schema:familyName Batra
    128 schema:givenName Dhruv
    129 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016177260105.00
    130 rdf:type schema:Person
    131 sg:person.07707707055.43 schema:affiliation https://www.grid.ac/institutes/grid.5386.8
    132 schema:familyName Gallagher
    133 schema:givenName Andrew
    134 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07707707055.43
    135 rdf:type schema:Person
    136 sg:pub.10.1007/11744047_13 schema:sameAs https://app.dimensions.ai/details/publication/pub.1031783594
    137 https://doi.org/10.1007/11744047_13
    138 rdf:type schema:CreativeWork
    139 sg:pub.10.1007/978-3-540-76390-1_57 schema:sameAs https://app.dimensions.ai/details/publication/pub.1037301069
    140 https://doi.org/10.1007/978-3-540-76390-1_57
    141 rdf:type schema:CreativeWork
    142 sg:pub.10.1007/978-3-540-88682-2_8 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045945012
    143 https://doi.org/10.1007/978-3-540-88682-2_8
    144 rdf:type schema:CreativeWork
    145 sg:pub.10.1007/978-3-642-33715-4_57 schema:sameAs https://app.dimensions.ai/details/publication/pub.1051335619
    146 https://doi.org/10.1007/978-3-642-33715-4_57
    147 rdf:type schema:CreativeWork
    148 sg:pub.10.1007/978-3-642-35740-4_17 schema:sameAs https://app.dimensions.ai/details/publication/pub.1014018594
    149 https://doi.org/10.1007/978-3-642-35740-4_17
    150 rdf:type schema:CreativeWork
    151 sg:pub.10.1007/s00530-002-0070-3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1026592439
    152 https://doi.org/10.1007/s00530-002-0070-3
    153 rdf:type schema:CreativeWork
    154 sg:pub.10.1007/s11263-006-0031-y schema:sameAs https://app.dimensions.ai/details/publication/pub.1043247765
    155 https://doi.org/10.1007/s11263-006-0031-y
    156 rdf:type schema:CreativeWork
    157 sg:pub.10.1007/s11263-007-0086-4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1015440148
    158 https://doi.org/10.1007/s11263-007-0086-4
    159 rdf:type schema:CreativeWork
    160 sg:pub.10.1007/s11263-010-0327-9 schema:sameAs https://app.dimensions.ai/details/publication/pub.1016062726
    161 https://doi.org/10.1007/s11263-010-0327-9
    162 rdf:type schema:CreativeWork
    163 sg:pub.10.1007/s11263-010-0415-x schema:sameAs https://app.dimensions.ai/details/publication/pub.1051801565
    164 https://doi.org/10.1007/s11263-010-0415-x
    165 rdf:type schema:CreativeWork
    166 sg:pub.10.1007/s11263-012-0537-4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018298508
    167 https://doi.org/10.1007/s11263-012-0537-4
    168 rdf:type schema:CreativeWork
    169 sg:pub.10.1023/a:1014573219977 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004426816
    170 https://doi.org/10.1023/a:1014573219977
    171 rdf:type schema:CreativeWork
    172 sg:pub.10.1023/b:visi.0000022288.19776.77 schema:sameAs https://app.dimensions.ai/details/publication/pub.1009092998
    173 https://doi.org/10.1023/b:visi.0000022288.19776.77
    174 rdf:type schema:CreativeWork
    175 sg:pub.10.1023/b:visi.0000025798.50602.3a schema:sameAs https://app.dimensions.ai/details/publication/pub.1036329061
    176 https://doi.org/10.1023/b:visi.0000025798.50602.3a
    177 rdf:type schema:CreativeWork
    178 https://doi.org/10.1006/cviu.1993.1030 schema:sameAs https://app.dimensions.ai/details/publication/pub.1054487278
    179 rdf:type schema:CreativeWork
    180 https://doi.org/10.1016/j.cviu.2006.07.011 schema:sameAs https://app.dimensions.ai/details/publication/pub.1013868408
    181 rdf:type schema:CreativeWork
    182 https://doi.org/10.1016/j.cviu.2008.07.002 schema:sameAs https://app.dimensions.ai/details/publication/pub.1008932526
    183 rdf:type schema:CreativeWork
    184 https://doi.org/10.1016/j.cviu.2011.02.011 schema:sameAs https://app.dimensions.ai/details/publication/pub.1041995597
    185 rdf:type schema:CreativeWork
    186 https://doi.org/10.1016/s0167-8655(02)00370-7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1026695583
    187 rdf:type schema:CreativeWork
    188 https://doi.org/10.1109/34.1000236 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061155588
    189 rdf:type schema:CreativeWork
    190 https://doi.org/10.1109/34.969114 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061157335
    191 rdf:type schema:CreativeWork
    192 https://doi.org/10.1109/cvpr.2009.5206651 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095223629
    193 rdf:type schema:CreativeWork
    194 https://doi.org/10.1109/cvpr.2010.5539802 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094846270
    195 rdf:type schema:CreativeWork
    196 https://doi.org/10.1109/cvpr.2010.5539804 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093859913
    197 rdf:type schema:CreativeWork
    198 https://doi.org/10.1109/cvpr.2010.5540055 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095170315
    199 rdf:type schema:CreativeWork
    200 https://doi.org/10.1109/cvpr.2010.5540193 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094320167
    201 rdf:type schema:CreativeWork
    202 https://doi.org/10.1109/cvpr.2011.5995530 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093473548
    203 rdf:type schema:CreativeWork
    204 https://doi.org/10.1109/cvpr.2011.5995638 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094630828
    205 rdf:type schema:CreativeWork
    206 https://doi.org/10.1109/iccv.1999.791253 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094521783
    207 rdf:type schema:CreativeWork
    208 https://doi.org/10.1109/iccv.2003.1238391 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093735574
    209 rdf:type schema:CreativeWork
    210 https://doi.org/10.1109/iccv.2007.4408844 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094384110
    211 rdf:type schema:CreativeWork
    212 https://doi.org/10.1109/iccv.2007.4408933 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095774143
    213 rdf:type schema:CreativeWork
    214 https://doi.org/10.1109/iccv.2009.5459145 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095132772
    215 rdf:type schema:CreativeWork
    216 https://doi.org/10.1109/iccv.2009.5459417 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095016346
    217 rdf:type schema:CreativeWork
    218 https://doi.org/10.1109/icip.2011.6116190 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095189133
    219 rdf:type schema:CreativeWork
    220 https://doi.org/10.1109/icpr.2008.4761602 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093884142
    221 rdf:type schema:CreativeWork
    222 https://doi.org/10.1109/tip.2008.924286 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061642123
    223 rdf:type schema:CreativeWork
    224 https://doi.org/10.1109/tpami.2004.1262177 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061742646
    225 rdf:type schema:CreativeWork
    226 https://doi.org/10.1109/tpami.2004.60 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061742742
    227 rdf:type schema:CreativeWork
    228 https://doi.org/10.1109/tpami.2008.132 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061743494
    229 rdf:type schema:CreativeWork
    230 https://doi.org/10.1109/tpami.2009.161 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061743739
    231 rdf:type schema:CreativeWork
    232 https://doi.org/10.1145/1073204.1073232 schema:sameAs https://app.dimensions.ai/details/publication/pub.1051131415
    233 rdf:type schema:CreativeWork
    234 https://doi.org/10.1145/1141911.1141964 schema:sameAs https://app.dimensions.ai/details/publication/pub.1063151975
    235 rdf:type schema:CreativeWork
    236 https://doi.org/10.1145/1179352.1141964 schema:sameAs https://app.dimensions.ai/details/publication/pub.1052911580
    237 rdf:type schema:CreativeWork
    238 https://doi.org/10.1145/1276377.1276485 schema:sameAs https://app.dimensions.ai/details/publication/pub.1063153865
    239 rdf:type schema:CreativeWork
    240 https://doi.org/10.1145/1409060.1409112 schema:sameAs https://app.dimensions.ai/details/publication/pub.1063155288
    241 rdf:type schema:CreativeWork
    242 https://doi.org/10.1145/1457515.1409112 schema:sameAs https://app.dimensions.ai/details/publication/pub.1029206453
    243 rdf:type schema:CreativeWork
    244 https://doi.org/10.1145/237170.237191 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045338552
    245 rdf:type schema:CreativeWork
    246 https://doi.org/10.5244/c.21.58 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099341553
    247 rdf:type schema:CreativeWork
    248 https://www.grid.ac/institutes/grid.419233.e schema:alternateName Siemens (United States)
    249 schema:name Siemens Corporation, Corporate Technology, Princeton, NJ, USA
    250 rdf:type schema:Organization
    251 https://www.grid.ac/institutes/grid.438526.e schema:alternateName Virginia Tech
    252 schema:name Virginia Tech, Blacksburg, VA, USA
    253 rdf:type schema:Organization
    254 https://www.grid.ac/institutes/grid.5386.8 schema:alternateName Cornell University
    255 schema:name Cornell University, Ithaca, NY, USA
    256 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...