Performance information

• Multi-depth HDR imaging system for improving accuracy in vein authentication　　　　　　　　　　　　　　　
  Yusuke Tanihata; Yu Feng; Keiichiro Kagawa; Kamel Mars; Keita Yasutomi; Shoji Kawahito; Takashi Komuro; Kazuya Nakano; Hiroyuki Suzuki
  Photonic Instrumentation Engineering XII, First page：32, Last page：32, Mar. 2025, [Reviewed]
  SPIE, International conference proceedings
  DOI：https://doi.org/10.1117/12.3040098
  DOI ID：10.1117/12.3040098
• High-Accuracy Acquisition of Both Hands’ Postures for Multiple Users Using Ultrafast Mirror-Drive Pan-Tilt Camera
  Yuta ABE; Taishi IRIYAMA; Takashi KOMURO; Kohei SHIMASAKI; Idaku ISHII
  Journal of the Japan Society for Precision Engineering, Volume：91, Number：3, First page：371, Last page：377, Mar. 2025, [Reviewed]
  Japan Society for Precision Engineering, Scientific journal
  DOI：https://doi.org/10.2493/jjspe.91.371
  DOI ID：10.2493/jjspe.91.371, ISSN：0912-0289, eISSN：1882-675X
• Anomaly Detection in Mutual Actions: Unsupervised Classification of Fighting and Non-Fighting Behaviors Using Transformer-Based Variational Autoencoder
  Thura Zaw; Takashi Komuro
  Lecture Notes in Computer Science, First page：397, Last page：410, Jan. 2025, [Reviewed]
  Springer Nature Switzerland, In book
  DOI：https://doi.org/10.1007/978-3-031-77392-1_30
  DOI ID：10.1007/978-3-031-77392-1_30, ISSN：0302-9743, eISSN：1611-3349
• Bit Depth Enhancement Considering Semantic Contextual Information via Spatial Feature Transform　　　　　　　　　　　　　　　
  Taishi Iriyama; Yuki Watanabe; Takashi Komuro
  IIEEJ Transactions on Image Electronics and Visual Computing, Volume：12, Number：2, First page：87, Last page：96, Dec. 2024, [Reviewed]
• An Asymmetric Multiplayer Augmented Reality Game with Spatial Sharing of a Physical Environment　　　　　　　　　　　　　　　
  Yuki Sawanobori; Taishi Iriyama; Takashi Komuro
  Proc. International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments (ICAT-EGVE 2024), Dec. 2024, [Reviewed]
• A Finger-Mounted Device for Shape and Tactile Perception in AR Mockups
  Soma Masuda; Taishi Iriyama; Takashi Komuro; Kenji Nakamura; Masato Takahashi
  2024 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), First page：385, Last page：386, Oct. 2024, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/ismar-adjunct64951.2024.00103
  DOI ID：10.1109/ismar-adjunct64951.2024.00103
• Self-measurement of 3D Leg Shape Using a Smartphone Through a Mirror　　　　　　　　　　　　　　　
  Shwe Yee Win; Takashi Komuro; Keiko Ogawa-Ochiai; Norimichi Tsumura
  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume：14862 LNCS, First page：290, Last page：299, Aug. 2024, [Reviewed]
  In this paper, we propose a self-measurement method of 3D leg shape for edema evaluation using only a smartphone and a mirror. In our method, patients can measure the 3D shape of their own leg by themselves at home. To obtain the initial shape of the leg, the user performs 3D scanning of a partial leg seen in a mirror. For the reconstruction of the unseen portion of the leg, ellipse fitting is applied to the slice images along the height of the leg. The 3D mesh of estimated full leg is reconstructed by connecting points on the ellipses. As the qualitative result, the estimated 3D mesh of leg shape from the partial leg seen through the mirror was obtained. For the accuracy measurement, we compared the estimated 3D leg shape with the original full leg shape by summing up the areas of the slice images for each estimated leg portion and the actual full leg portion as the quantitative result. It can be seen that our proposed method can reconstruct the 3D shape of whole leg from the partial portion of the leg seen through the mirror with sufficient accuracy.
  International conference proceedings
  DOI：https://doi.org/10.1007/978-981-97-5578-3_23
  Scopus：https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85202600993&origin=inward
  Scopus Citedby：https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=85202600993&origin=inward
  DOI ID：10.1007/978-981-97-5578-3_23, ISSN：0302-9743, eISSN：1611-3349, SCOPUS ID：85202600993
• A Mid-air Multi-touch Interface using an Ultrafast Pan-tilt Camera
  Yuta Abe; Taishi Iriyama; Takashi Komuro; Kohei Shimasaki; Idaku Ishii
  Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, Volume：2016, First page：1, Last page：6, May 2024, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3613905.3652007
  DOI ID：10.1145/3613905.3652007
• PSp-Transformer: A Transformer with Data-level Probabilistic Sparsity for Action Representation Learning
  Jiaxin Zhou; Takashi Komuro
  ITE Transactions on Media Technology and Applications, Volume：12, Number：1, First page：123, Last page：132, Jan. 2024, [Reviewed]
  Institute of Image Information and Television Engineers, Scientific journal
  DOI：https://doi.org/10.3169/mta.12.123
  DOI ID：10.3169/mta.12.123, eISSN：2186-7364
• Comparison of Autoencoder Models for Unsupervised Representation Learning of Skeleton Sequences
  Thura Zaw; Jiaxin Zhou; Takashi Komuro
  Lecture Notes in Computer Science, First page：135, Last page：146, Dec. 2023, [Reviewed]
  Springer Nature Switzerland, In book
  DOI：https://doi.org/10.1007/978-3-031-47969-4_11
  DOI ID：10.1007/978-3-031-47969-4_11, ISSN：0302-9743, eISSN：1611-3349
• View interpolation networks for reproducing the material appearance of specular objects　　　　　　　　　　　　　　　
  Chihiro Hoshizawa; Takashi Komuro
  Virtual Reality & Intelligent Hardware, Volume：5, Number：1, First page：1, Last page：10, Feb. 2023, [Reviewed]
  Elsevier BV, Scientific journal
  DOI：https://doi.org/10.1016/j.vrih.2022.11.001
  DOI ID：10.1016/j.vrih.2022.11.001, ISSN：2096-5796
• AR Digital Workspace Using a Mobile Device
  Yuki Kojima; Taishi Iriyama; Takashi Komuro
  Proceedings of the 2022 ACM Symposium on Spatial User Interaction, Volume：2017, First page：1, Last page：2, Dec. 2022, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3565970.3567690
  DOI ID：10.1145/3565970.3567690
• Visualization of Parkinson’s Disease Tremor for a Telemedicine System
  Tingyu Du; Takashi Komuro; Keiko Ogawa-Ochiai
  Communications in Computer and Information Science, First page：414, Last page：419, Nov. 2022, [Reviewed]
  Springer Nature Switzerland, In book
  DOI：https://doi.org/10.1007/978-3-031-19679-9_52
  DOI ID：10.1007/978-3-031-19679-9_52, ISSN：1865-0929, eISSN：1865-0937
• High-speed image generation of transparent objects using an image conversion network
  Sana Nishinaga; Karin Kaifuku; Jingtao Huang; Takashi Komuro; Kayo Ogawa
  2022 IEEE 11th Global Conference on Consumer Electronics (GCCE), First page：736, Last page：737, Oct. 2022, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/gcce56475.2022.10014072
  DOI ID：10.1109/gcce56475.2022.10014072
• 3D Hand Pose Recognition Over a Wide Area Using Two Omnidirectional Cameras with Field-of-view Division
  Yuta Abe; Takashi Komuro
  Lecture Notes in Computer Science, First page：3, Last page：17, Oct. 2022, [Reviewed]
  Springer Nature Switzerland, In book
  DOI：https://doi.org/10.1007/978-3-031-17618-0_1
  DOI ID：10.1007/978-3-031-17618-0_1, ISSN：0302-9743, eISSN：1611-3349
• Lightweight Wearable AR System using Head-mounted Projector for Work Support
  Haruki Yuda; Masamichi Limori; Taishi Iriyama; Takashi Komuro
  2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Volume：31, First page：690, Last page：691, Oct. 2022, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/ismar-adjunct57072.2022.00144
  DOI ID：10.1109/ismar-adjunct57072.2022.00144
• An asymmetrical-structure auto-encoder for unsupervised representation learning of skeleton sequences　　　　　　　　　　　　　　　
  Jiaxin Zhou; Takashi Komuro
  Computer Vision and Image Understanding, Volume：222, First page：103491, Last page：103491, Sep. 2022, [Reviewed]
  Elsevier BV, Scientific journal
  DOI：https://doi.org/10.1016/j.cviu.2022.103491
  DOI ID：10.1016/j.cviu.2022.103491, ISSN：1077-3142
• Blockwise Feature-Based Registration of Deformable Medical Images
  Su Wai Tun; Takashi Komuro; Hajime Nagahara
  Lecture Notes in Computer Science, First page：472, Last page：482, Aug. 2022, [Reviewed]
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-031-13870-6_40
  DOI ID：10.1007/978-3-031-13870-6_40, ISSN：0302-9743, eISSN：1611-3349
• Resolving Multi-Path Interference in Compressive Time-of-Flight Depth Imaging with a Multi-Tap Macro-Pixel Computational CMOS Image Sensor
  Masaya Horio; Yu Feng; Tomoya Kokado; Taishi Takasawa; Keita Yasutomi; Shoji Kawahito; Takashi Komuro; Hajime Nagahara; Keiichiro Kagawa
  Sensors, Volume：22, Number：7, First page：2442, Last page：2442, Mar. 2022, [Reviewed]
  Multi-path interference causes depth errors in indirect time-of-flight (ToF) cameras. In this paper, resolving multi-path interference caused by surface reflections using a multi-tap macro-pixel computational CMOS image sensor is demonstrated. The imaging area is implemented by an array of macro-pixels composed of four subpixels embodied by a four-tap lateral electric field charge modulator (LEFM). This sensor can simultaneously acquire 16 images for different temporal shutters. This method can reproduce more than 16 images based on compressive sensing with multi-frequency shutters and sub-clock shifting. In simulations, an object was placed 16 m away from the sensor, and the depth of an interference object was varied from 1 to 32 m in 1 m steps. The two reflections were separated in two stages: coarse estimation based on a compressive sensing solver and refinement by a nonlinear search to investigate the potential of our sensor. Relative standard deviation (precision) and relative mean error (accuracy) were evaluated under the influence of photon shot noise. The proposed method was verified using a prototype multi-tap macro-pixel computational CMOS image sensor in single-path and dual-path situations. In the experiment, an acrylic plate was placed 1 m or 2 m and a mirror 9.3 m from the sensor.
  MDPI AG, Scientific journal
  DOI：https://doi.org/10.3390/s22072442
  DOI ID：10.3390/s22072442, eISSN：1424-8220
• Users’ Content Memorization in Multi-User Interactive Public Displays
  Narumi Sugiura; Rikako Ogura; Yoshio Matsuda; Takashi Komuro; Kayo Ogawa
  International Journal of Human–Computer Interaction, Volume：38, Number：5, First page：447, Last page：455, Mar. 2022, [Reviewed]
  Informa UK Limited, Scientific journal
  DOI：https://doi.org/10.1080/10447318.2021.1948686
  DOI ID：10.1080/10447318.2021.1948686, ISSN：1044-7318, eISSN：1532-7590
• Investigation of Advertising Performance by Image Effects in Interactive Digital Signage
  Shiori Ino; Takashi Komuro; Kayo Ogawa
  The Journal of The Institute of Image Information and Television Engineers, Volume：76, Number：2, First page：297, Last page：303, Mar. 2022, [Reviewed]
  Institute of Image Information and Television Engineers, Scientific journal
  DOI：https://doi.org/10.3169/itej.76.297
  DOI ID：10.3169/itej.76.297, ISSN：1342-6907, eISSN：1881-6908
• AR Peephole Interface: Extending the workspace of a mobile device using real-space information　　　　　　　　　　　　　　　
  Masashi Miyazaki; Takashi Komuro
  Pervasive and Mobile Computing, Volume：78, First page：101489, Last page：101489, Dec. 2021, [Reviewed]
  Elsevier BV, Scientific journal
  DOI：https://doi.org/10.1016/j.pmcj.2021.101489
  DOI ID：10.1016/j.pmcj.2021.101489, ISSN：1574-1192
• 3D Hand Pointing Recognition over a Wide Area using Two Fisheye Cameras
  Azusa Kuramochi; Takashi Komuro
  Lecture Notes in Computer Science, First page：58, Last page：67, Nov. 2021, [Reviewed]
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-030-90963-5_5
  DOI ID：10.1007/978-3-030-90963-5_5, ISSN：0302-9743, eISSN：1611-3349
• Finger region extraction using color of skin for hand-waving finger vein authentication　　　　　　　　　　　　　　　
  Hiroyuki Suzuki; Narissa Ditthapakdijanya; Takashi Komuro; Keiichiro Kagawa; Kazuya Nakano; Takashi Obi
  Biomedical Imaging and Sensing Conference 2021, First page：33, Last page：33, Oct. 2021, [Reviewed]
  SPIE, International conference proceedings
  DOI：https://doi.org/10.1117/12.2615690
  DOI ID：10.1117/12.2615690
• Reproduction of Environment Reflection using Extrapolation of Front Camera Images in Mobile AR
  Shun Odajima; Takashi Komuro
  2021 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Oct. 2021, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/ismar-adjunct54149.2021.00068
  DOI ID：10.1109/ismar-adjunct54149.2021.00068
• Transmission of correct gaze direction in video conferencing using screen-embedded cameras
  Kazuki Kobayashi; Takashi Komuro; Keiichiro Kagawa; Shoji Kawahito
  Multimedia Tools and Applications, Volume：80, Number：21-23, First page：31509, Last page：31526, Sep. 2021, [Reviewed]
  Springer Science and Business Media LLC, Scientific journal
  DOI：https://doi.org/10.1007/s11042-020-09758-w
  DOI ID：10.1007/s11042-020-09758-w, ISSN：1380-7501, eISSN：1573-7721
• Human detection from low-resolution video images using 3D convolutional neural network　　　　　　　　　　　　　　　
  Hiroki Kanazawa; Yuta Nakamoto; Jiaxin Zhou; Takashi Komuro
  Fifteenth International Conference on Quality Control by Artificial Vision, Volume：1, First page：52, Last page：52, Jul. 2021, [Reviewed]
  SPIE, International conference proceedings
  DOI：https://doi.org/10.1117/12.2589829
  DOI ID：10.1117/12.2589829
• Grasping Action Recognition in VR Environment using Object Shape and Position Information
  Koki Hirota; Takashi Komuro
  2021 IEEE International Conference on Consumer Electronics (ICCE), First page：1, Last page：2, Jan. 2021, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/icce50685.2021.9427608
  DOI ID：10.1109/icce50685.2021.9427608
• BRDF Measurement of Real Materials Using Handheld Cameras
  Haru Otani; Takashi Komuro
  Lecture Notes in Computer Science, First page：65, Last page：77, 2021, [Reviewed]
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-030-90439-5_6
  DOI ID：10.1007/978-3-030-90439-5_6, ISSN：0302-9743, eISSN：1611-3349
• Wearable Augmented Reality System Using Head-Mounted Projector
  Masamichi Iimori; Takashi Komuro
  Lecture Notes in Computer Science, First page：481, Last page：491, 2021, [Reviewed]
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-030-90439-5_38
  DOI ID：10.1007/978-3-030-90439-5_38, ISSN：0302-9743, eISSN：1611-3349
• Generation of Virtual Reality Environment Based on 3D Scanned Indoor Physical Space
  Satoshi Moro; Takashi Komuro
  Lecture Notes in Computer Science, First page：492, Last page：503, 2021, [Reviewed]
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-030-90439-5_39
  DOI ID：10.1007/978-3-030-90439-5_39, ISSN：0302-9743, eISSN：1611-3349
• 3D Registration of Deformable Objects Using a Time-of-Flight Camera
  Su Wai Tun; Takashi Komuro; Hajime Nagahara
  Lecture Notes in Computer Science, First page：455, Last page：465, 2021, [Reviewed]
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-030-90439-5_36
  DOI ID：10.1007/978-3-030-90439-5_36, ISSN：0302-9743, eISSN：1611-3349
• Removal of Floating Particles from Underwater Images Using Image Transformation Networks
  Lei Li; Takashi Komuro; Koichiro Enomoto; Masashi Toda
  Pattern Recognition. ICPR International Workshops and Challenges, Volume：12662 LNCS, First page：414, Last page：421, 2021, [Reviewed]
  In this paper, we propose three methods for removing floating particles from underwater images. The first two methods are based on Generative Adversarial Networks (GANs). The first method uses CycleGAN which can be trained with an unpaired dataset, and the second method uses pix2pixHD that is trained with a paired dataset created by adding artificial particles to underwater images. The third method consists of two-step process – particle detection and image inpainting. For particle detection, an image segmentation neural network U-Net is trained by using underwater images added with artificial particles. Using the output of U-Net, the particle regions are repaired by an image inpainting network Partial Convolutions. The experimental results showed that the methods using GANs were able to remove floating particles, but the resolution became lower than that of the original images. On the other hand, the results of the method using U-Net and Partial Convolutions showed that it is capable of accurate detection and removal of floating particles without loss of resolution.
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-030-68790-8_32
  Scopus：https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85110679867&origin=inward
  Scopus Citedby：https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=85110679867&origin=inward
  DOI ID：10.1007/978-3-030-68790-8_32, ISSN：0302-9743, eISSN：1611-3349, SCOPUS ID：85110679867
• Recognizing Gestures from Videos using a Network with Two-branch Structure and Additional Motion Cues
  Jiaxin Zhou; Takashi Komuro
  2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020), Nov. 2020, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/fg47880.2020.00006
  DOI ID：10.1109/fg47880.2020.00006
• A Portable Measurement System for Spatially-Varying Reflectance Using Two Handheld Cameras
  Zar Zar Tun; Seiji Tsunezaki; Takashi Komuro; Shoji Yamamoto; Norimichi Tsumura
  Lecture Notes in Computer Science, First page：266, Last page：276, Oct. 2020, [Reviewed]
  Springer International Publishing, In book
  DOI：https://doi.org/10.1007/978-3-030-59990-4_20
  DOI ID：10.1007/978-3-030-59990-4_20, ISSN：0302-9743, eISSN：1611-3349
• Dynamic layout optimization for multi-user interaction with a large display
  Yoshio Matsuda; Takashi Komuro
  Proceedings of the 25th International Conference on Intelligent User Interfaces, Mar. 2020, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3377325.3377481
  DOI ID：10.1145/3377325.3377481
• Palm-Controlled Pointing Interface Using a Dynamic Photometric Stereo Camera.　　　　　　　　　　　　　　　
  Yoshio Matsuda; Takashi Komuro; Takuya Yoda; Hajime Nagahara; Shoji Kawahito; Keiichiro Kagawa
  HCI International 2020 - Late Breaking Posters - 22nd International Conference, First page：142, Last page：147, 2020, [Reviewed]
  Springer, International conference proceedings
  DOI：https://doi.org/10.1007/978-3-030-60700-5_19
  DOI ID：10.1007/978-3-030-60700-5_19, DBLP ID：conf/hci/MatsudaKYNKK20
• Object Manipulation for Perceiving a Sense of Material using User-Perspective Mobile Augmented Reality
  Ryota Nomura; Takashi Komuro; Shoji Yamamoto; Norimichi Tsumura
  ITE Transactions on Media Technology and Applications, Volume：8, Number：4, First page：245, Last page：251, 2020, [Reviewed]
  Institute of Image Information and Television Engineers, Scientific journal
  DOI：https://doi.org/10.3169/mta.8.245
  DOI ID：10.3169/mta.8.245, eISSN：2186-7364
• Situation-Adaptive Object Grasping Recognition in VR Environment
  Koki Hirota; Takashi Komuro
  2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), Dec. 2019, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/aivr46125.2019.00035
  DOI ID：10.1109/aivr46125.2019.00035
• Semi-Automatic Creation of an Anime-Like 3D Face Model from a Single Illustration
  Takayuki Niki; Takashi Komuro
  2019 International Conference on Cyberworlds (CW), Volume：32, First page：53, Last page：56, Oct. 2019, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/cw.2019.00017
  DOI ID：10.1109/cw.2019.00017
• Augmented Reality-Based Peephole Interaction using Real Space Information
  Masashi Miyazaki; Takashi Komuro
  2019 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Oct. 2019, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/ismar-adjunct.2019.00017
  DOI ID：10.1109/ismar-adjunct.2019.00017
• Recognizing Fall Actions from Videos Using Reconstruction Error of Variational Autoencoder
  Jiaxin Zhou; Takashi Komuro
  2019 IEEE International Conference on Image Processing (ICIP), Sep. 2019, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/icip.2019.8803671
  DOI ID：10.1109/icip.2019.8803671
• Simultaneous interaction with a large display by many users
  Yuriko Sakakibara; Yoshio Matsuda; Takashi Komuro; Kayo Ogawa
  Proceedings of the 8th ACM International Symposium on Pervasive Displays, Jun. 2019, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3321335.3329680
  DOI ID：10.1145/3321335.3329680
• Technology for visualizing the local change in shape of edema using a depth camera
  Kenta Masui; Kaoru Kiyomitsu; Keiko Ogawa-Ochiai; Takashi Komuro; Norimichi Tsumura
  Artificial Life and Robotics, Volume：24, Number：4, First page：480, Last page：486, Jun. 2019, [Reviewed]
  Springer Science and Business Media LLC, Scientific journal
  DOI：https://doi.org/10.1007/s10015-019-00541-1
  DOI ID：10.1007/s10015-019-00541-1, ISSN：1433-5298, eISSN：1614-7456
• Bivariate BRDF Estimation Based on Compressed Sensing　　　　　　　　　　　　　　　
  Haru Otani; Takashi Komuro; Shoji Yamamoto; Norimichi Tsumura
  Lecture Notes in Computer Science (LNCS), Volume：11542, First page：483, Last page：489, 2019, [Reviewed]
  We propose a method of estimating a bivariate BRDF from a small number of sampled data using compressed sensing. This method aims to estimate the reflectance of various materials by using the representation space that keeps local information when restored by compressed sensing. We conducted simulated measurements using randomly sampled data and data sampled according to the camera position and orientation, and confirmed that most of the BRDF was successfully restored from 40% sampled data in the case of simulated measurement using a camera and markers.
  English, International conference proceedings
  DOI：https://doi.org/10.1007/978-3-030-22514-8_48
  Scopus：https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85067667705&origin=inward
  Scopus Citedby：https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=85067667705&origin=inward
  DOI ID：10.1007/978-3-030-22514-8_48, ISSN：0302-9743, eISSN：1611-3349, SCOPUS ID：85067667705
• Measuring Reflectance of Anisotropic Materials Using Two Handheld Cameras　　　　　　　　　　　　　　　
  Zar Zar Tun; Seiji Tsunezaki; Takashi Komuro; Shoji Yamamoto; Norimichi Tsumura
  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume：11844 LNCS, First page：482, Last page：493, 2019, [Reviewed]
  In this paper, we propose a method for measuring the reflectance of anisotropic materials using a simple apparatus consisting of two handheld cameras, a small LED light source, a turning table and a chessboard with markers. The system is configured to obtain the different incoming and outgoing light directions, and the brightness of pixels on the surface of the material. The anisotropic Ward BRDF (Bidirectional Reflectance Distribution Function) model is used to approximate the reflectance, and the model parameters are estimated from the incoming and outgoing angles and the brightness of pixels by using a non-linear optimization method. The initial values of the anisotropic direction are given based on the peak specular lobe on the surface, and the best-fitted one is chosen for the anisotropic direction. The optimized parameters show the well-fitted results between the observed brightness and the BRDF model for each RGB channel. It was confirmed that our system was able to measure the reflectance of different isotropic and anisotropic materials.
  International conference proceedings
  DOI：https://doi.org/10.1007/978-3-030-33720-9_37
  Scopus：https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85076128969&origin=inward
  Scopus Citedby：https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=85076128969&origin=inward
  DOI ID：10.1007/978-3-030-33720-9_37, ISSN：0302-9743, eISSN：1611-3349, SCOPUS ID：85076128969
• On-mouse projector: Peephole interaction using a mouse with a mobile projector　　　　　　　　　　　　　　　
  Tomohiro Araki; Takashi Komuro
  Pervasive and Mobile Computing, Volume：50, First page：124, Last page：136, Oct. 2018, [Reviewed]
  Elsevier BV, Scientific journal
  DOI：https://doi.org/10.1016/j.pmcj.2018.09.002
  DOI ID：10.1016/j.pmcj.2018.09.002, ISSN：1574-1192
• A Large Effective Touchscreen Using a Head-Mounted Projector
  Yusuke Kemmoku; Takashi Komuro
  Information, Volume：9, Number：9, First page：235, Last page：235, Sep. 2018, [Reviewed]
  In our previous work, we proposed a user interface in which a user wears a projector and a depth camera on his or her head and performs touch operations on an image projected on a flat surface. By using the head-mounted projector, images are always projected in front of the user in the direction of the user’s gaze. The image to be projected is changed according to the user’s head pose so as to fix the superimposed image on the surface, which realizes a large effective screen size. In this paper, we conducted an experiment for evaluating the accuracy of registration by measuring the positional and rotational errors between the real world and the superimposed image using our experimental system. As a result, the mean absolute errors of translation were about 10 mm when the user stopped his head, and the delay was estimated to be about 0.2 s. We also discuss the limitations of our prototype and show the direction of future development.
  MDPI AG, Scientific journal
  DOI：https://doi.org/10.3390/info9090235
  DOI ID：10.3390/info9090235, eISSN：2078-2489
• A Multi-user Interactive Public Display with Dynamic Layout Optimization
  Yoshio Matsuda; Takashi Komuro
  Proceedings of the 7th ACM International Symposium on Pervasive Displays, First page：1, Last page：2, Jun. 2018, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3205873.3210712
  DOI ID：10.1145/3205873.3210712
• Multi-Aperture-Based Probabilistic Noise Reduction of Random Telegraph Signal Noise and Photon Shot Noise in Semi-Photon-Counting Complementary-Metal-Oxide-Semiconductor Image Sensor
  Haruki Ishida; Keiichiro Kagawa; Takashi Komuro; Bo Zhang; Min-Woong Seo; Taishi Takasawa; Keita Yasutomi; Shoji Kawahito
  Sensors, Volume：18, Number：4, First page：977, Last page：977, Mar. 2018, [Reviewed]
  A probabilistic method to remove the random telegraph signal (RTS) noise and to increase the signal level is proposed, and was verified by simulation based on measured real sensor noise. Although semi-photon-counting-level (SPCL) ultra-low noise complementary-metal-oxide-semiconductor (CMOS) image sensors (CISs) with high conversion gain pixels have emerged, they still suffer from huge RTS noise, which is inherent to the CISs. The proposed method utilizes a multi-aperture (MA) camera that is composed of multiple sets of an SPCL CIS and a moderately fast and compact imaging lens to emulate a very fast single lens. Due to the redundancy of the MA camera, the RTS noise is removed by the maximum likelihood estimation where noise characteristics are modeled by the probability density distribution. In the proposed method, the photon shot noise is also relatively reduced because of the averaging effect, where the pixel values of all the multiple apertures are considered. An extremely low-light condition that the maximum number of electrons per aperture was the only 2 e − was simulated. PSNRs of a test image for simple averaging, selective averaging (our previous method), and the proposed method were 11.92 dB, 11.61 dB, and 13.14 dB, respectively. The selective averaging, which can remove RTS noise, was worse than the simple averaging because it ignores the pixels with RTS noise and photon shot noise was less improved. The simulation results showed that the proposed method provided the best noise reduction performance.
  MDPI AG, Scientific journal
  DOI：https://doi.org/10.3390/s18040977
  DOI ID：10.3390/s18040977, eISSN：1424-8220
• Extended Workspace using a Smartphone with a Depth Camera　　　　　　　　　　　　　　　
  Masashi Miyazaki; Takashi Komuro
  Adj. Proc. 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2018), First page：115, Last page：116, 2018, [Reviewed]
  English, International conference proceedings
• Reproducing Material Appearance of Real Objects using Mobile Augmented Reality　　　　　　　　　　　　　　　
  Seiji Tsunezaki; Ryota Nomura; Takashi Komuro; Shoji Yamamoto; Norimichi Tsumura
  Adj. Proc. 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2018), First page：196, Last page：197, 2018, [Reviewed]
  English, International conference proceedings
• On-mouse projector: Peephole Interaction using a Mouse with a Projector
  Tomohiro Araki; Takashi Komuro
  Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia, Nov. 2017, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3152832.3152849
  DOI ID：10.1145/3152832.3152849, SCOPUS ID：85040508502
• Comparative study on text entry methods for mobile devices with a hover function
  Toshiaki Aiyoshizawa; Takashi Komuro
  Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia, First page：355, Last page：361, Nov. 2017, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3152832.3156614
  DOI ID：10.1145/3152832.3156614
• Recognition of typing motions on AR typing interface
  Masae Okada; Masakazu Higuchi; Takashi Komuro; Kayo Ogawa
  Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia, First page：429, Last page：434, Nov. 2017, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3152832.3156618
  DOI ID：10.1145/3152832.3156618
• A gaze-preserving group video conference system using screen-embedded cameras　　　　　　　　　　　　　　　
  Kazuki Kobayashi; Takashi Komuro; Bo Zhang; Keiichiro Kagawa; Shoji Kawahito
  Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST, Volume：131944, First page：82, Nov. 2017, [Reviewed]
  In this paper, we propose a system that presents correct gaze directions of each user in a group-to-group video conference. It is known that gaze direction plays a significant role in face-to-face communication. However, in ordinary video conference systems, there is a problem that gaze directions cannot be correctly transmitted to a user in the other side since the camera is placed on the top or bottom of the display. We propose a method to generate images of each person from an appropriate viewpoint position using Kinect and screen-embedded cameras and to present the images to the user. By estimating each user's gaze direction and rendering the image of the person who the user is looking at from the users' viewpoint position, it is possible to present an image suitable for each user even in a group-to-group situation. We show demonstration of a group-to-group video conference using the proposed system and show that the system can present images suitable for each user.
  Association for Computing Machinery, English, International conference proceedings
  DOI：https://doi.org/10.1145/3139131.3141775
  DOI ID：10.1145/3139131.3141775, SCOPUS ID：85038562897
• Distant Pointing User Interfaces based on 3D Hand Pointing Recognition
  Yutaka Endo; Dai Fujita; Takashi Komuro
  Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces, Oct. 2017, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/3132272.3132292
  DOI ID：10.1145/3132272.3132292
• VolGrab: Realizing 3D view navigation by aerial hand gestures　　　　　　　　　　　　　　　
  Shun Sekiguchi; Takashi Komuro
  Conference on Human Factors in Computing Systems - Proceedings, Volume：127655, First page：2064, Last page：2069, May 2017, [Reviewed]
  In this paper, we propose a user interface that combines the existing touch interface with 3D view navigation by aerial hand gestures and that realizes simultaneous zooming and scrolling. While the touch interface enables only planar operation, aerial hand gestures enable 3D operation with an increased number of axes, and enables smooth navigation that integrates zoom and scroll operations by assigning the direction that is normal to the screen to zooming and the direction that is parallel to the screen to scrolling. We implemented a map application based on the proposed interface to confirm the effectiveness of the proposed interface.
  Association for Computing Machinery, English, International conference proceedings
  DOI：https://doi.org/10.1145/3027063.3053108
  DOI ID：10.1145/3027063.3053108, SCOPUS ID：85019650259
• Accurate Distance Estimation of a Preceding Vehicle Using High-speed Stereo Video Images
  Ming Li; Takashi Komuro
  IEEJ Transactions on Electronics, Information and Systems, Volume：137, Number：7, First page：916, Last page：917, 2017, [Reviewed]
  Institute of Electrical Engineers of Japan (IEE Japan), Scientific journal
  DOI：https://doi.org/10.1541/ieejeiss.137.916
  DOI ID：10.1541/ieejeiss.137.916, ISSN：0385-4221, eISSN：1348-8155
• Tracking and removal of suspended matter from underwater video images.　　　　　　　　　　　　　　　
  Takashi Komuro; Kun Chen; Koichiro Enomoto; Masashi Toda; Naoaki Tezuka
  Proceedings of SPIE - The International Society for Optical Engineering, Volume：10338, First page：103380, 2017, [Reviewed]
  We propose a method for obtaining clear underwater images by tracking the motion of suspended matter from video images captured in water and by separating the images into foreground and background. We assume that input images are the superposition of a foreground and a background, and constructed a transition model and the observation model. An input image is divided into patches and tracking of the foreground in each patch is performed while applying Kalman filter to separate the input images into the foreground and the background. From the result of the experiment using simulated images, we confirmed that the background images were successfully estimated and a region that was moving slowly was also recognized as a part of the background.
  SPIE-INT SOC OPTICAL ENGINEERING, English, International conference proceedings
  DOI：https://doi.org/10.1117/12.2264066
  Scopus：https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85020274915&origin=inward
  Scopus Citedby：https://www.scopus.com/inward/citedby.uri?partnerID=HzOxMe3b&scp=85020274915&origin=inward
  DOI ID：10.1117/12.2264066, ISSN：0277-786X, eISSN：1996-756X, DBLP ID：conf/qcav/KomuroCETT17, SCOPUS ID：85020274915, Web of Science ID：WOS:000404646500029
• Mobile augmented reality for providing perception of materials.
  Ryota Nomura; Yuko Unuma; Takashi Komuro; Shoji Yamamoto; Norimichi Tsumura
  Proceedings of the 16th International Conference on Mobile and Ubiquitous Multimedia, MUM 2017, Stuttgart, Germany, November 26 - 29, 2017, First page：501, Last page：506, 2017, [Reviewed]
  ACM
  DBLP ID：conf/mum/NomuraUKYT17
• 3D interaction with virtual objects in a precisely-aligned view using a see-through mobile AR system　　　　　　　　　　　　　　　
  Yuko Unuma; Takashi Komuro
  ITE Transactions on Media Technology and Applications, Volume：5, Number：2, First page：49, Last page：56, 2017, [Reviewed]
  In this paper, we propose a system that enables users to interact with virtual objects that are displayed on a mobile display in a precisely-aligned view using their hands. By projecting a 3D scene obtained by a depth camera according to the user's viewpoint position, the scene including the user's hand displayed on the screen appears seamlessly connected to the actual scene outside the screen, which enables natural interaction with virtual objects through the screen. We conducted an experiment to evaluate the positional accuracy in the presented images. The maximum mean error was 8.60 mm, and the maximum standard deviation was 1.69 mm, which could be improved by further refinement. We also conducted an experiment to evaluate the usability of the system. We asked the participants to perform tasks using the proposed system in the aligned and non-aligned see-through modes. Despite some restrictions in our prototype system, 9 out of 14 participants completed the task faster in the aligned see-through mode. This result shows the future potential of the proposed system in interaction with virtual objects.
  Institute of Image Information and Television Engineers, English, Scientific journal
  DOI：https://doi.org/10.3169/mta.5.49
  DOI ID：10.3169/mta.5.49, ISSN：2186-7364, SCOPUS ID：85016820545
• 3D pose estimation of a front-pointing hand using a random regression forest　　　　　　　　　　　　　　　
  Dai Fujita; Takashi Komuro
  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Volume：10118, First page：197, Last page：211, 2017, [Reviewed]
  In this paper, we propose a method for estimating the 3D poses of a front-pointing hand from camera images to realize freehand pointing interaction from a distance. Our method uses a Random Regression Forest (RRF) to realize robust estimation against environmental and individual variations. In order to improve the estimation accuracy, our method supports the use of two cameras and integrates the distributions of the hand poses for these cameras, which are modeled by the Gaussian mixture model. Moreover, tracking of the hand poses further improves the estimation accuracy and stability. The results of performance evaluation showed that the root mean square error of the angle estimation was 4.10°, which is accurate enough to expect that our proposed method can be applied to user interface systems.
  Springer Verlag, English, International conference proceedings
  DOI：https://doi.org/10.1007/978-3-319-54526-4_15
  DOI ID：10.1007/978-3-319-54526-4_15, ISSN：1611-3349, SCOPUS ID：85016100129
• PZBoard: A Prediction-based Zooming Interface for Supporting Text Entry on a Mobile Device
  Toshiaki Aiyoshizawa; Naoto Koarai; Takashi Komuro
  Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, First page：627, Last page：632, Sep. 2016, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/2957265.2961828
  DOI ID：10.1145/2957265.2961828
• On-mouse projector: Extended workspace using a mouse with a projector　　　　　　　　　　　　　　　
  Tomohiro Araki; Takashi Komuro
  Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, MobileHCI 2016, First page：605, Last page：610, Sep. 2016, [Reviewed]
  In this paper, we propose On-mouse projector, an interface that combines a mouse and a mobile projector. This system satisfies both portability and large information space and enables stable operation. The mobile projector is placed on the mouse and projects images on a surface in front of the mouse. The projected image presents a part of large information space and users can change the area to see by moving the mouse. The system is assumed to be used on a flat surface such as a desk and users can stably perform the same operation as the ordinary mouse operation. We created a prototype in which a projector is fixed above a mouse using acrylic plates. This prototype works in a standalone configuration by using a stick PC and realizes various operations such as object selection, object moving and image zooming.
  Association for Computing Machinery, Inc, English, International conference proceedings
  DOI：https://doi.org/10.1145/2957265.2961829
  DOI ID：10.1145/2957265.2961829, SCOPUS ID：84991096397
• Dynamic Information Space Based on High-Speed Sensor Technology.　　　　　　　　　　　　　　　
  Masatoshi Ishikawa; Idaku Ishii; Yutaka Sakaguchi; Makoto Shimojo; Hiroyuki Shinoda; Hirotsugu; Yamamoto; Takashi Komuro; Array,Array; Yoshihiro Watanabe
  Human-Harmonized Information Technology, Volume 1 - Vertical Impact, First page：97, Last page：136, 2016, [Reviewed]
  Springer
  DOI：https://doi.org/10.1007/978-4-431-55867-5_5
  DOI ID：10.1007/978-4-431-55867-5_5, DBLP ID：books/sp/16/IshikawaISSSYKONW16
• AR Tabletop Interface Using an Optical See-Through HMD　　　　　　　　　　　　　　　
  Nozomi Sugiura; Takashi Komuro
  SUI'16: PROCEEDINGS OF THE 2016 SYMPOSIUM ON SPATIAL USER INTERACTION, First page：173, Last page：173, 2016, [Reviewed]
  We propose a user interface that superimposes a virtual touch panel on a flat surface using an optical see-through head-mounted display and an RGB-D camera. The user can use the interface in a hands-free state, and can perform the operation with both hands. The interface performs markerless superimposition of virtual objects on a real scene. In addition, the interface can recognize three-dimensional information of the user's fingers, allowing the user to operate with the virtual touch panel. We developed some applications in which the user can perform various operations on the virtual touch panel.
  ASSOC COMPUTING MACHINERY, English, International conference proceedings
  DOI：https://doi.org/10.1145/2983310.2989180
  DOI ID：10.1145/2983310.2989180, Web of Science ID：WOS:000387817100029
• Space-sharing AR Interaction on Multiple Mobile Devices with a Depth Camera　　　　　　　　　　　　　　　
  Yuki Kaneto; Takashi Komuro
  2016 IEEE VIRTUAL REALITY CONFERENCE (VR), First page：197, Last page：198, 2016, [Reviewed]
  In this paper, we propose a markerless augmented reality (AR) system that works on multiple mobile devices. The relative positions and orientations of the devices and their individual motions are estimated from 3D information in real space obtained by depth cameras attached to the devices. The system allows multiple users to share the AR space and to interact with the same virtual object. To estimate the relative positions and orientations of the devices, the system generates 2D images by looking down from above at the 3D scene obtained by the depth cameras, performs 2D registration using template matching, and obtains a transformation matrix that transforms the coordinate system of one camera to that of another camera. The motion of a camera is estimated using the ICP algorithm to realize markerless AR. Using the proposed system, we created an application that enables multiple users to interact with the same virtual object.
  IEEE, English, International conference proceedings
  DOI：https://doi.org/10.1109/VR.2016.7504721
  DOI ID：10.1109/VR.2016.7504721, ISSN：1087-8270, Web of Science ID：WOS:000386307300039
• AR Tabletop Interface using a Head-Mounted Projector　　　　　　　　　　　　　　　
  Yusuke Kemmoku; Takashi Komuro
  ADJUNCT PROCEEDINGS OF THE 2016 IEEE INTERNATIONAL SYMPOSIUM ON MIXED AND AUGMENTED REALITY (ISMAR-ADJUNCT), First page：288, Last page：291, 2016, [Reviewed]
  In this paper, we propose a tabletop interface in which a user wears a projector with a depth camera on his or her head and can perform touch operations on an image projected on a flat surface. By using the head-mounted projector, images are always projected in front of the user in the direction of the user's gaze. By changing the image to be projected based on the user's head movement, this interface realizes a large effective screen size. The system superimposes an image on the flat surface by performing plane detection, placing the image on the detected plane, performing perspective projection to obtain a 2D image, and projecting the 2D image using the projector. Registration between the real world and the image is performed by estimating the user's head pose using the detected plane information. Furthermore, touch input is recognized by detecting the user's finger on the plane using the depth camera. We implemented some application examples into the system to demonstrate the usefulness of the proposed interface.
  IEEE, English, International conference proceedings
  DOI：https://doi.org/10.1109/ISMAR-Adjunct.2016.88
  DOI ID：10.1109/ISMAR-Adjunct.2016.88, Web of Science ID：WOS:000401904700073
• 3D Tabletop User Interface with High Synchronization Accuracy using a High-speed Stereo Camera　　　　　　　　　　　　　　　
  Takumi Kusano; Takashi Komuro
  Proc. the 2015 ACM International Conference on Interactive Tabletops and Surfaces (ITS 2015), First page：39, Last page：42, Dec. 2015, [Reviewed]
  English, International conference proceedings
• A tabletop projector-camera system for remote and nearby pointing operation　　　　　　　　　　　　　　　
  Shun Sekiguchi; Takashi Komuro
  Conference on Human Factors in Computing Systems - Proceedings, Volume：18, First page：1621, Last page：1626, Apr. 2015, [Reviewed]
  In this paper, we propose a pointing interface that can be easily installed and that enables remote and near by operations with high recognition accuracy. We use a system put on a table consisting of multiple projectors and depth cameras. The system projects a shared screen and a personal screen per each user on a projection screen and on a table respectively and enables interaction according to the user's hand motion. A personal screen can be operated by touch operations and a shared screen can be operated by pointing with fingers. Touch and pointing direction are recognized from the depth image and the confidence map obtained by a depth camera. We developed an early prototype system for verification and implemented a presentation application that supports collaborative work to the system. This application allows a user to move a window between the shared screen and the personal screen and to perform various operations on the slides. Copyright is held by the author/owner(s).
  Association for Computing Machinery, English, International conference proceedings
  DOI：https://doi.org/10.1145/2702613.2732731
  DOI ID：10.1145/2702613.2732731, SCOPUS ID：84954243721
• Multi-finger AR typing interface for mobile devices using high-speed hand motion recognition　　　　　　　　　　　　　　　
  Masakazu Higuchi; Takashi Komuro
  Conference on Human Factors in Computing Systems - Proceedings, Volume：18, First page：1235, Last page：1240, Apr. 2015, [Reviewed]
  In this paper, we propose a new user interface designed to improve the usability of mobile devices. The interface overlays a virtual keyboard on real images captured by a camera attached to the back of a mobile device and recognizes user's hand motions by using optical flow information. The user can operate the virtual keyboard with his/her multiple fingers in the space behind the mobile device. The interface recognizes only hand motions and does not recognize poses of a hand or fingers, which enables high-speed stable recognition without complicated processing. The experimental system recognized hand motions with a processing time of about 8.33 [ms/frame] (about 120 [fps]). We implemented a keyboard application using our interface on the experimental system and evaluated the performance. The result of a simple user study showed that typing speed became faster as a trial is repeated. Copyright is held by the author/owner(s).
  Association for Computing Machinery, English, International conference proceedings
  DOI：https://doi.org/10.1145/2702613.2732741
  DOI ID：10.1145/2702613.2732741, SCOPUS ID：84954230577
• Three-dimensional VR Interaction Using the Movement of a Mobile Display　　　　　　　　　　　　　　　
  Lili Wang; Takashi Komuro
  2015 IEEE VIRTUAL REALITY CONFERENCE (VR), First page：363, Last page：364, 2015, [Reviewed]
  In this study, we propose a VR system for allowing various types of interaction with virtual objects using an autostereoscopic mobile display and an accelerometer. The system obtains the orientation and motion information from the accelerometer attached to the mobile display and reflects them to the motion of virtual objects. It can present 3D images with motion parallax by estimating the position of the user's viewpoint and by displaying properly projected images. Furthermore, our method enables to connect the real space and the virtual space seamlessly through the mobile display by determining the coordinate system so that one of the horizontal surfaces in the virtual space coincides with the display surface. To show the effectiveness of this concept, we implemented an application to simulate food cooking by regarding the mobile display as a frying pan.
  IEEE COMPUTER SOC, English, International conference proceedings
  DOI：https://doi.org/10.1109/VR.2015.7223446
  DOI ID：10.1109/VR.2015.7223446, ISSN：1087-8270, Web of Science ID：WOS:000371409700128
• Overlaying Navigation Signs on a Road Surface using a Head-Up Display　　　　　　　　　　　　　　　
  Kaho Ueno; Takashi Komuro
  2015 IEEE International Symposium on Mixed and Augmented Reality, First page：168, Last page：169, 2015, [Reviewed]
  In this paper, we propose a method for overlaying navigation signs on a road surface and displaying them on a head-up display (HUD). Accurate overlaying is realized by measuring 3D data of the surface in real time using a depth camera. In addition, the effect of head movement is reduced by performing face tracking with a camera that is placed in front of the HUD, and by performing distortion correction of projection images according to the driver's viewpoint position. Using an experimental system, we conducted an experiment to display a navigation sign and confirmed that the sign is overlaid on a surface. We also confirmed that the sign looks to be fixed on the surface in real space.
  IEEE, English, International conference proceedings
  DOI：https://doi.org/10.1109/ISMAR.2015.48
  DOI ID：10.1109/ISMAR.2015.48, Web of Science ID：WOS:000380611200038
• Three-Dimensional Hand Pointing Recognition Using Two Cameras by Interpolation and Integration of Classification Scores　　　　　　　　　　　　　　　
  Dai Fujita; Takashi Komuro
  COMPUTER VISION - ECCV 2014 WORKSHOPS, PT I, Volume：8925, First page：713, Last page：726, 2015, [Reviewed]
  In this paper, we propose a novel method of hand recognition for remote mid-air pointing operation. In the proposed method, classification scores are calculated in a sliding window for hand postures with different pointing directions. Detection of a pointing hand and estimation of the pointing direction is performed by interpolating the classification scores. Moreover, we introduce two cameras and improve the recognition accuracy by integrating the classification scores obtained from two camera images. In the experiment, the recognition rate was 73% at around 1 FPPI when +/- 10 degrees. error was allowed. Though this result was still insufficient for practical applications, we confirmed that integration of two camera information greatly improved the recognition performance.
  SPRINGER-VERLAG BERLIN, English, International conference proceedings
  DOI：https://doi.org/10.1007/978-3-319-16178-5_50
  DOI ID：10.1007/978-3-319-16178-5_50, ISSN：0302-9743, Web of Science ID：WOS:000362493800050
• Dynamic 3D Interaction using an Optical See-through HMD　　　　　　　　　　　　　　　
  Nozomi Sugiura; Takashi Komuro
  2015 IEEE VIRTUAL REALITY CONFERENCE (VR), First page：359, Last page：360, 2015, [Reviewed]
  We propose a system that enables dynamic 3D interaction with real and virtual objects using an optical see-through head-mounted display and an RGB-D camera. The virtual objects move according to physical laws. The system uses a physics engine for calculation of the motion of virtual objects and collision detection. In addition, the system performs collision detection between virtual objects and real objects in the three-dimensional scene obtained from the camera which is dynamically updated. A user wears the device and interacts with virtual objects in a seated position. The system gives users a great sense of reality through an interaction with virtual objects.
  IEEE COMPUTER SOC, English, International conference proceedings
  DOI：https://doi.org/10.1109/VR.2015.7223444
  DOI ID：10.1109/VR.2015.7223444, ISSN：1087-8270, Web of Science ID：WOS:000371409700126
• Natural 3D Interaction using a See-through Mobile AR System　　　　　　　　　　　　　　　
  Yuko Unuma; Takashi Komuro
  2015 IEEE International Symposium on Mixed and Augmented Reality, First page：84, Last page：87, 2015, [Reviewed]
  In this paper, we propose an interaction system in which the appearance of the image displayed on a mobile display is consistent with that of the real space and that enables a user to interact with virtual objects overlaid on the image using the user's hand. The three-dimensional scene obtained by a depth camera is projected according to the user's viewpoint position obtained by face tracking, and the see-through image whose appearance is consistent with that outside the mobile display is generated. Interaction with virtual objects is realized by using the depth information obtained by the depth camera. To move virtual objects as if they were in real space, virtual objects are rendered in the world coordinate system that is fixed to a real scene even if the mobile display moves, and the direction of gravitational force added to virtual objects is made consistent with that of the world coordinate system. The former is realized by using the ICP ( Iterative Closest Point) algorithm and the latter is realized by using the information obtained by an accelerometer. Thus, natural interaction with virtual objects using the user's hand is realized.
  IEEE, English, International conference proceedings
  DOI：https://doi.org/10.1109/ISMAR.2015.27
  DOI ID：10.1109/ISMAR.2015.27, Web of Science ID：WOS:000380611200017
• 3D Tabletop User Interface Using Virtual Elastic Objects
  Hiroaki Tateyama; Takumi Kusano; Takashi Komuro
  Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, First page：283, Last page：288, Nov. 2014, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/2669485.2669533
  DOI ID：10.1145/2669485.2669533
• Evaluation of Visuo-haptic Feedback in a 3D Touch Panel Interface
  Xu Zhao; Takehiro Niikura; Takashi Komuro
  Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces, First page：299, Last page：304, Nov. 2014, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/2669485.2669536
  DOI ID：10.1145/2669485.2669536
• In-Air Finger Motion Interface for Mobile Devices with Vibration Feedback　　　　　　　　　　　　　　　
  Takehiro Niikura; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  IEEJ TRANSACTIONS ON ELECTRICAL AND ELECTRONIC ENGINEERING, Volume：9, Number：4, First page：375, Last page：383, Jul. 2014, [Reviewed]
  In this paper, we propose the 'in-air finger motion interface', which solves a problem users face in interacting with mobile devices. The interface uses only a single small camera to track the three-dimensional (3D) position of the user's finger. The finger region is robustly extracted by using flashing infrared light-emitting diodes (LEDs) and by obtaining the difference image between an illuminated image and a nonilluminated image. The movement of the finger is modeled by transformation of a finger image, and its parameters are accurately estimated by iterative template matching. As a result of the evaluation, an operating area 131% larger than the size of the display and accuracy of input action of 0.6 mm in the x-direction are realized. Finally, to investigate the effect of vibration feedback, we conducted a study and found that vibration feedback significantly reduces the average time to finish the task as well as the number of input errors. (C) 2014 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.
  WILEY-BLACKWELL, English, Scientific journal
  DOI：https://doi.org/10.1002/tee.21982
  DOI ID：10.1002/tee.21982, ISSN：1931-4973, eISSN：1931-4981, Web of Science ID：WOS:000337798600005
• Real-time typing action detection in a 3D pointing gesture interface
  Risa Ishijima; Kayo Ogawa; Masakazu Higuchi; Takashi Komuro
  Proceedings of the 5th Augmented Human International Conference, First page：1, Last page：2, Mar. 2014, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/2582051.2582071
  DOI ID：10.1145/2582051.2582071
• Multi-finger AR typing interface for mobile devices
  Satoshi Sagara; Masakazu Higuchi; Takashi Komuro
  Proceedings of the companion publication of the 19th international conference on Intelligent User Interfaces, First page：13, Last page：16, Feb. 2014, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/2559184.2559197
  DOI ID：10.1145/2559184.2559197
• See-through mobile ar system for natural 3d interaction　　　　　　　　　　　　　　　
  Yuko Unuma; Takehiro Niikura; Takashi Komuro
  International Conference on Intelligent User Interfaces, Proceedings IUI, First page：17, Last page：20, 2014, [Reviewed]
  In this paper, we propose an interaction system which displays see-through images on the mobile display and that allows a user to interact with virtual objects overlaid on the see-through image using the user's hand. In this system, the camera which tracks the user's viewpoint is attached to the front of the mobile display and the depth camera which captures color and depth images of the user's hand and the background scene is attached to the back of the mobile display. Natural interaction with virtual objects using the user's hand is realized by displaying images so that the appearance of a space through the mobile display is consistent with that of the real space from the user's viewpoint. We implemented two applications to the system and showed the usefulness of this system in various AR applications.
  Association for Computing Machinery, English, International conference proceedings
  DOI：https://doi.org/10.1145/2559184.2559198
  DOI ID：10.1145/2559184.2559198, SCOPUS ID：84898968628
• Object Tracking and Image Restoration from Multi-Frame Images Captured in a Dark Environment　　　　　　　　　　　　　　　
  Takashi Komuro; Hayato Kuroda
  ITE Transactions on Media Technology and Applications, Volume：2, Number：2, First page：176, Last page：184, 2014, [Reviewed]
  We show a method of realizing object tracking and image restoration in the dark in which target motion and a reference image are simultaneously estimated using a Bayesian framework. To avoid being trapped in a local minimum in the gradient calculation, a broader search is performed by calculating differences after applying a strong low-pass filter to input images. Deblurring is performed using the motion parameters estimated from the blurred images. As a result, we realized object tracking and image restoration from simulated video images with an SNR of up to -6 dB, and real video images captured in a dark environment of less than 0.05 lx illuminance at the subject surface. In addition, we examined the optimal frame rate for image restoration and we found that a higher frame rate was better under relatively little noise while a lower frame rate was better under much noise.
  Institute of Image Information and Television Engineers, English, Scientific journal
  DOI：https://doi.org/10.3169/mta.2.176
  DOI ID：10.3169/mta.2.176, ISSN：2186-7364, SCOPUS ID：84952064936
• AR typing interface for mobile devices
  Masakazu Higuchi; Takashi Komuro
  Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia, First page：1, Last page：8, Dec. 2013, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/2541831.2541847
  DOI ID：10.1145/2541831.2541847
• A Zooming Interface for Accurate Text Input on Mobile Devices　　　　　　　　　　　　　　　
  Naoto Koarai; Takashi Komuro
  Ext. Abst. ACM SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), First page：1299, Last page：1304, May 2013, [Reviewed]
  English, International conference proceedings
• Geometrically consistent mobile AR for 3D interaction
  Hikari Uchida; Takashi Komuro
  Proceedings of the 4th Augmented Human International Conference, First page：229, Last page：230, Mar. 2013, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/2459236.2459275
  DOI ID：10.1145/2459236.2459275
• A virtually tangible 3D interaction system using an autostereoscopic display　　　　　　　　　　　　　　　
  Takumi Kusano; Takehiro Niikura; Takashi Komuro
  SUI 2013 - Proceedings of the ACM Symposium on Spatial User Interaction, First page：87, 2013, [Reviewed]
  We propose a virtually tangible 3D interaction system that enables direct interaction with three dimensional virtual objects which are presented on an autostereoscopic display. Copyright © ACM 2013.
  English, International conference proceedings
  DOI：https://doi.org/10.1145/2491367.2491394
  DOI ID：10.1145/2491367.2491394, SCOPUS ID：84882945851
• Robust finger tracking for gesture control of mobile devices using contour and interior information of a finger　　　　　　　　　　　　　　　
  Masakazu Higuchi; Takashi Komuro
  ITE Transactions on Media Technology and Applications, Volume：1, Number：3, First page：226, Last page：236, 2013, [Reviewed]
  In this paper, we propose a method for robust tracking of a moving finger in an image sequence. The method is suitable for application to our input interface system, which recognizes a moving finger in the air. The proposed method extracts edges from input images, and then estimates the position and rotation of a finger in the input images by matching points in a template to edges. The most remarkable feature of our method is that it also takes into account the presence or absence of edges in regions in the input images corresponding to the inside of the finger in the template for estimating. This makes it possible to estimate the position and rotation of a finger exactly in images with complex backgrounds. Our method successfully tracked a finger in several situations with an average processing time of 6.32 [ms/frame], and the finger was tracked with good accuracy.
  Institute of Image Information and Television Engineers, English, Scientific journal
  DOI：https://doi.org/10.3169/mta.1.226
  DOI ID：10.3169/mta.1.226, ISSN：2186-7364, SCOPUS ID：84899865110
• Input action classification in a 3D gesture interface for mobile devices
  Kayo Ogawa; Naoko Sakata; Tomoko Muraiso; Takashi Komuro
  The 1st IEEE Global Conference on Consumer Electronics 2012, Volume：21, First page：418, Last page：421, Oct. 2012, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/gcce.2012.6379644
  DOI ID：10.1109/gcce.2012.6379644
• Poster: Zooming interface using a 3D finger position for mobile devices
  R. Fujii; T. Komuro
  2012 IEEE Symposium on 3D User Interfaces (3DUI), First page：143, Last page：144, Mar. 2012, [Reviewed]
  IEEE, International conference proceedings
  DOI：https://doi.org/10.1109/3dui.2012.6184202
  DOI ID：10.1109/3dui.2012.6184202
• High-speed estimation of multi-finger position and pose for input interface of the mobile devices　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Atsushi Matsutani; Takehiro Niikura; Takashi Komuro; Masatoshi Ishikawa
  1st IEEE Global Conference on Consumer Electronics 2012, GCCE 2012, First page：228, Last page：232, 2012, [Reviewed]
  Mobile devices are too small to operate freely using their input surfaces. To solve this problem, non-contact and natural gesture interfaces have been the focus of recent research. In this paper we propose a method of estimating multi-finger position and pose for operating such devices at high speed using a single camera. Our method achieves the finger tracking based on the appearance and shape deformation model by estimating the translational movements and the degree of bent finger. The experimental results show that our method can obtain the position of the hand and the pose of the each finger within 9.7 ms. © 2012 IEEE.
  English, International conference proceedings
  DOI：https://doi.org/10.1109/GCCE.2012.6379588
  DOI ID：10.1109/GCCE.2012.6379588, SCOPUS ID：84872335440
• Finger detection based on data fusion of fish-eye stereo camera for 3D-gesture input interface　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Shintaro Kubo; Takashi Komuro; Masatoshi Ishikawa
  1st IEEE Global Conference on Consumer Electronics 2012, GCCE 2012, First page：279, Last page：283, 2012, [Reviewed]
  There is a growing importance of the in-air hand operation for the input devices. In order to introduce such in-air operation, we require an image sensing technology that enables 3D position detection of the user's hand in wide range. In this paper, as a practical application system, we focus on the Zooming Touch Panel [1] that connects the touch-panel and in-air operations seamlessly. In order to provide wide operation space for this type of the system, we propose a new hand detection method based on data fusion of fish-eye stereo camera. This technique enables both high-accuracy and wide-range 3D hand detection. Also this paper shows two example demonstrations. © 2012 IEEE.
  English, International conference proceedings
  DOI：https://doi.org/10.1109/GCCE.2012.6379602
  DOI ID：10.1109/GCCE.2012.6379602, SCOPUS ID：84872311818
• Human Gait Estimation Using a Wearable Camera(Real World Imaging)　　　　　　　　　　　　　　　
  Watanabe Yoshihiro; Hatanaka Tetsuo; Komuro Takashi; Ishikawa Masatoshi
  Transactions of the Virtual Reality Society of Japan, Volume：17, Number：3, First page：219, Last page：229, 2012
  We focus on the growing need for a technology that can achieve motion capture in outdoor environments. The conventional approaches have relied mainly on fixed installed cameras. With this approach, however, it is difficult to capture motion in everyday surroundings. This paper describes a new method for motion estimation using a single wearable camera. We focused on walking motion. The key point is how the system can estimate the original walking state using limited information from a wearable sensor. This paper describes three aspects: the configuration of the sensing system, gait representation, and the gait estimation method.
  THE VIRTUAL REALITY SOCIETY OF JAPAN, Japanese
  DOI：https://doi.org/10.18974/tvrsj.17.3_219
  DOI ID：10.18974/tvrsj.17.3_219, ISSN：1344-011X, CiNii Articles ID：110009517477, CiNii Books ID：AA11448578
• 3D touch panel interface using an autostereoscopic display　　　　　　　　　　　　　　　
  Takehiro Niikura; Takashi Komuro
  ITS 2012 - Proceedings of the ACM Conference on Interactive Tabletops and Surfaces, First page：295, Last page：298, 2012, [Reviewed]
  We propose a 3D touch panel interface using an autostereoscopic display and a high-speed stereo camera. With this system, the virtual objects are stereoscopically-presented, and the objects respond to the hand movement captured by a stereo camera, which makes users feel like they are touching the objects directly. Since we used high-speed camera for detecting the fingertip, it can realize more accurate synchronization between the real object and virtual object without a feeling of strangeness. © 2012 Authors.
  English, International conference proceedings
  DOI：https://doi.org/10.1145/2396636.2396683
  DOI ID：10.1145/2396636.2396683, SCOPUS ID：84871264160
• In-air Typing Interface: Realizing 3D operation for mobile devices　　　　　　　　　　　　　　　
  Takehiro Niikura; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  1st IEEE Global Conference on Consumer Electronics 2012, GCCE 2012, First page：223, Last page：227, 2012, [Reviewed]
  In this paper, we explore the usability of the "In-air Typing Interface" that we developed to overcome the problems common with traditional mobile interfaces. We performed two studies focused on frame rate, latency, and tactile feedback. We found that reducing the latency even by several milliseconds can improve the operating efficiency, and the frame rate partly affects the efficiency. Tactile feedback significantly reduced the number of mistypes, and it is likely that vibration feedback can improve the operating efficiency further if there is a penalty for erroneous operations. © 2012 IEEE.
  English, International conference proceedings
  DOI：https://doi.org/10.1109/GCCE.2012.6379587
  DOI ID：10.1109/GCCE.2012.6379587, SCOPUS ID：84872284516
• Human gait estimation using a wearable camera　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Tetsuo Hatanaka; Takashi Komuro; Masatoshi Ishikawa
  2011 IEEE Workshop on Applications of Computer Vision, WACV 2011, First page：276, Last page：281, 2011, [Reviewed]
  We focus on the growing need for a technology that can achieve motion capture in outdoor environments. The conventional approaches have relied mainly on fixed installed cameras. With this approach, however, it is difficult to capture motion in everyday surroundings. This paper describes a new method for motion estimation using a single wearable camera. We focused on walking motion. The key point is how the system can estimate the original walking state using limited information from a wearable sensor. This paper describes three aspects: the configuration of the sensing system, gait representation, and the gait estimation method. © 2010 IEEE.
  English, International conference proceedings
  DOI：https://doi.org/10.1109/WACV.2011.5711514
  DOI ID：10.1109/WACV.2011.5711514, SCOPUS ID：79952523102
• Stereo 3D Reconstruction using Prior Knowledge of Indoor Scenes　　　　　　　　　　　　　　　
  Kentaro Kofuji; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  2011 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), First page：5198, Last page：5203, 2011, [Reviewed]
  We propose a new method of indoor-scene stereo vision that uses probabilistic prior knowledge of indoor scenes in order to exploit the global structure of artificial objects. In our method, we assume three properties of the global structure - planarity, connectivity, and parallelism/orthogonality - and we formulate them in the framework of maximum a posteriori (MAP) estimation. To enable robust estimation, we employ a probability distribution that has both high peaks and wide flat tails. In experiments, we demonstrated that our approach can estimate shapes whose surfaces are not constrained by three orthogonal planes. Furthermore, comparing our results with those of a conventional method that assumes a locally smooth disparity map suggested that the proposed method can estimate more globally consistent shapes.
  IEEE, English, International conference proceedings
  DOI：https://doi.org/10.1109/ICRA.2011.5979560
  DOI ID：10.1109/ICRA.2011.5979560, ISSN：1050-4729, Web of Science ID：WOS:000324383404069
• Surface image synthesis of moving spinning cans using a 1,000-fps area scan camera　　　　　　　　　　　　　　　
  Tomohira Tabata; Takashi Komuro; Masatoshi Ishikawa
  MACHINE VISION AND APPLICATIONS, Volume：21, Number：5, First page：643, Last page：652, Aug. 2010, [Reviewed]
  We demonstrate surface-image synthesis of moving, spinning cylindrical objects using a commercially available high-speed area scan camera. The frame rate used in the demonstration experiment was 1,000 fps, which is sufficient to achieve surface-image synthesis of cylinders spun at up to 36 rps. We successfully demonstrated a technique based on an algorithm similar to image mosaicing at 1,000 fps, for the first time to the best of our knowledge. In this paper, we discuss techniques to overcome the potential problems faced when applying surface-image synthesis to cylindrical objects, such as image distortion, quantization errors due to superimposing images, and intensity variations due to the surface curvature. An FPGA-based parallel image processing board, PB-1, that we developed was used to implement these demonstrations. We introduce this application of PB-1 as a potential practical solution to the long-standing problem of industrial visual inspection using real-time high-speed vision.
  SPRINGER, English, Scientific journal
  DOI：https://doi.org/10.1007/s00138-010-0247-2
  DOI ID：10.1007/s00138-010-0247-2, ISSN：0932-8092, eISSN：1432-1769, Web of Science ID：WOS:000280249300005
• A Reconfigurable Embedded System for 1000 f/s Real-Time Vision　　　　　　　　　　　　　　　
  Takashi Komuro; Tomohira Tabata; Masatoshi Ishikawa
  IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, Volume：20, Number：4, First page：496, Last page：504, Apr. 2010, [Reviewed]
  In this paper, we proposed an architecture of embedded systems for high-frame-rate real-time vision on the order of 1000 f/s, which achieved both hardware reconfigurability and easy algorithm implementation while fulfilling performance demands. The proposed system consisted of an embedded microprocessor and. field programmable gate arrays (FPGAs). A coprocessor consisting of memory units, direct memory access controller units, and image processing units were implemented in each FPGA. While the number of units and functions are reconfigurable by reprogramming the FPGAs, users can implement algorithms without hardware knowledge. A descriptor method in which the central processing unit gave instructions to each coprocessor through a register array enabled task-level parallel processing as well as pixel-level parallel processing in the processing units. The specifications of an evaluation system developed based on the proposed architecture, the results of performance evaluation, and application examples using the system were shown.
  IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, English, Scientific journal
  DOI：https://doi.org/10.1109/TCSVT.2009.2035832
  DOI ID：10.1109/TCSVT.2009.2035832, ISSN：1051-8215, eISSN：1558-2205, Web of Science ID：WOS:000276681000003
• Estimation of non-rigid surface deformation using developable surface model　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Nakashima; Takashi Komuro; Masatoshi Ishikawa
  Proceedings - International Conference on Pattern Recognition, First page：197, Last page：200, 2010, [Reviewed]
  There is a strong demand for a method of acquiring a non-rigid shape under deformation with high accuracy and high resolution. However, this is difficult to achieve because of performance limitations in measurement hardware. In this paper, we propose a model-based method for estimating non-rigid deformation of a developable surface. The model is based on geometric characteristics of the surface, which are important in various applications. This method improves the accuracy of surface estimation and planar development from a low-resolution point cloud. Experiments using curved documents showed the effectiveness of the proposed method. © 2010 IEEE.
  English, International conference proceedings
  DOI：https://doi.org/10.1109/ICPR.2010.57
  DOI ID：10.1109/ICPR.2010.57, ISSN：1051-4651, SCOPUS ID：78149492478
• Interactive Display System based on Adaptive Image Projection to a Deformable Tangible Screen　　　　　　　　　　　　　　　
  Watanabe Yoshihiro; Cassinelli Alvaro; Komuro Takashi; Ishikawa Masatoshi
  Transactions of the Virtual Reality Society of Japan, Volume：15, Number：2, First page：173, Last page：182, 2010, [Reviewed]
  We propose a variant of the multi-touch display technology that introduces an original way of manipulating three-dimensional data. The underlying metaphor is that of a deformable screen that acts as a boundary surface between the real and the virtual worlds. By doing so, the interface can create the illusion of continuity between the user's real space and the virtual three-dimensional space. The developed system presented here enables this by employing three key technologies: a tangible and deformable projection screen, a real-time three-dimensional sensing mechanism, and an algorithm for dynamic compensation for anamorphic projection. This paper introduces the concept of the deformable tangible workspace, and describes the required technologies for implementing it. Also, several applications developed on a prototype system are detailed and demonstrated.
  THE VIRTUAL REALITY SOCIETY OF JAPAN, Japanese
  DOI：https://doi.org/10.18974/tvrsj.15.2_173
  DOI ID：10.18974/tvrsj.15.2_173, ISSN：1344-011X, CiNii Articles ID：110008722701, CiNii Books ID：AA11448578
• In-air typing interface for mobile devices with vibration feedback　　　　　　　　　　　　　　　
  Takehiro Niikura; Yuki Hirobe; Alvaro Cassinelli; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  ACM SIGGRAPH 2010 Emerging Technologies, SIGGRAPH '10, 2010, [Reviewed]
  Recently the miniaturization of mobile devices has progressed and such devices are difficult to have input interface that has wide operation area on their surface. Conventional input interface on a cell phone, such as a touch panel or keypad, has limited operation area. There has been many approaches to handle this problem, but they require users to wear some physical devices[Harrison and Hudson 2009] or to use in some specific environments[Roeber et al. 2003].
  English, International conference proceedings
  DOI：https://doi.org/10.1145/1836821.1836836
  DOI ID：10.1145/1836821.1836836, SCOPUS ID：77956298100
• WIDE RANGE IMAGE SENSING USING A THROWN-UP CAMERA　　　　　　　　　　　　　　　
  Toshitaka Kuwa; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  2010 IEEE INTERNATIONAL CONFERENCE ON MULTIMEDIA AND EXPO (ICME 2010), First page：878, Last page：883, 2010, [Reviewed]
  In this paper, we propose a wide-range image sensing method using a camera thrown up into the air. By using camera thrown up in this way, we can get images that are otherwise difficult to obtain, such as those taken from overhead. As an example of wide-range image sensing, we integrated video images captured by a thrown-up camera using an image mosaicing technique. When rotation about the optical axis of the camera can be ignored, we can integrate images by mosaicing using a translational approximation, which preferentially pastes pixels around the image center. To obtain the information about the camera direction, a rotational approximation using the angles of incident light rays is required. We also propose use of a high-framerate camera (HFR camera) in order to acquire a large amount of information. A seamless large image was obtained by synthesizing the images captured by a thrown-up HFR camera. We found that high frame rates of around 1000 fps were necessary.
  IEEE, English, International conference proceedings
  ISSN：1945-7871, Web of Science ID：WOS:000287977700155
• A QVGA-SIZE PIXEL-PARALLEL IMAGE PROCESSOR FOR 1,000-FPS VISION　　　　　　　　　　　　　　　
  Takashi Komuro; Atsushi Iwashita; Masatoshi Ishikawa
  IEEE MICRO, Volume：29, Number：6, First page：58, Last page：67, Nov. 2009, [Reviewed]
  MASSIVELY PARALLEL IMAGE PROCESSING WITH AS MANY PROCESSING ELEMENTS AS PIXELS CAN ACHIEVE REAL-TIME VISION WITH RATES AS HIGH AS 1,000 FRAMES PER SECOND. THE AUTHORS IMPLEMENTED A QVGA-SIZE PIXEL-PARALLEL IMAGE PROCESSOR FOR OBJECT IDENTIFICATION AND POSE ESTIMATION ON A SINGLE CHIP. BIT-SERIAL OPERATION AND DYNAMIC LOGIC REDUCE THE CIRCUIT AREA, AND PIPELINING ENABLES HIGH PROCESSING SPEED.
  IEEE COMPUTER SOC, English, Scientific journal
  DOI：https://doi.org/10.1109/MM.2009.89
  DOI ID：10.1109/MM.2009.89, ISSN：0272-1732, eISSN：1937-4143, Web of Science ID：WOS:000273096300008
• Vision-based Input Interface for Mobile Devices with High-speed Fingertip Tracking　　　　　　　　　　　　　　　
  Yuki Hirobe; Takehiro Niikura; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  Adjunct Proceedings of 22nd ACM Symposium on User Interface Software and Technology (UIST 2009), First page：7, Last page：8, Oct. 2009, [Reviewed]
  English, International conference proceedings
• Ptz control with head tracking for video chat
  Kota Yamaguchi; Takashi Komuro; Masatoshi Ishikawa
  CHI '09 Extended Abstracts on Human Factors in Computing Systems, First page：3919, Last page：3924, Apr. 2009, [Reviewed]
  ACM, International conference proceedings
  DOI：https://doi.org/10.1145/1520340.1520594
  DOI ID：10.1145/1520340.1520594
• High-resolution Shape Reconstruction from Multiple Range Images based on Simultaneous Estimation of Surface and Motion　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  2009 IEEE 12TH INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), First page：1787, Last page：1794, 2009, [Reviewed]
  Recognition of dynamic scenes based on shape information could be useful for various applications. In this study, we aimed at improving the resolution of three-dimensional (3D) data obtained from moving targets. We present a simple clean and robust method that jointly estimates motion parameters and a high-resolution 3D shape. Experimental results are provided to illustrate the performance of the proposed algorithm.
  IEEE, English, International conference proceedings
  ISSN：1550-5499, Web of Science ID：WOS:000294955300230
• Fast finger tracking system for in-air typing interface　　　　　　　　　　　　　　　
  Kazuhiro Terajima; Takashi Komuro; Masatoshi Ishikawa
  Conference on Human Factors in Computing Systems - Proceedings, First page：3739, Last page：3744, 2009, [Reviewed]
  We developed a system which performs 3D motion tracking of human's hand and fingers from images of a single high-frame-rate camera and that recognizes his/her typing motion in the air. Our template-matching-based method using hand textures reduces background effect and enables markerless tracking. In addition, use of a high-frame-rate camera enables recognition of rapid typing motion which is difficult to track using standard cameras. In order to realize realtime recognition, we developed hardware which parallelizes and accelerates image processing. As a result, we achieved real-time recognition of typing motion with the throughput of 138 fps (frames per second) and the latency of 29 ms.
  English, International conference proceedings
  DOI：https://doi.org/10.1145/1520340.1520564
  DOI ID：10.1145/1520340.1520564, SCOPUS ID：70349183566
• The Deformable Workspace: a Membrane between Real and Virtual Space　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Alvaro Cassinelli; Takashi Komuro; Masatoshi Ishikawa
  THIRD ANNUAL IEEE INTERNATIONAL WORKSHOP ON HORIZONTAL INTERACTIVE HUMAN-COMPUTER SYSTEMS, PROCEEDINGS, First page：155, Last page：162, 2008, [Reviewed]
  We propose a variant of the multi-touch display technology that introduces an original way of manipulating three-dimensional data. The underlying metaphor is that of a deformable screen that acts as a boundary surface between the real and the virtual worlds. By doing so, the interface can create the illusion of continuity between the user's real space and the virtual three-dimensional space. The prototype system presented here enables this by employing three key technologies: a tangible and deformable projection screen, a real-time three-dimensional sensing mechanism, and an algorithm for dynamic compensation for anamorphic projection. This paper introduces the concept of the deformable tangible workspace, and describes the required technologies for implementing it. Also, several applications developed on a prototype system are detailed and demonstrated.
  IEEE, English, International conference proceedings
  Web of Science ID：WOS:000264096100022
• Interleaved Pixel Lookup for Embedded Computer Vision　　　　　　　　　　　　　　　
  Kota Yamaguchi; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  2008 IEEE COMPUTER SOCIETY CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, VOLS 1-3, First page：1470, Last page：1477, 2008, [Reviewed]
  This paper describes an in-depth investigation and implementation of interleaved memory for pixel lookup operations in computer vision. Pixel lookup, mapping between coordinates and pixels, is a common operation in computer vision, but is also a potential bottleneck due to formidable bandwidth requirements for real-time operation. We focus on the acceleration of pixel lookup operations through parallelizing memory banks by interleaving. The key to applying interleaving for pixel lookup is 2D block data partitioning and support for unaligned access. With this optimization of interleaving, pixel lookup operations can output a block of pixels at once without major overhead for unaligned access. An example implementation of our optimized interleaved memory for affine motion tracking shows that the pixel lookup operations can achieve 12.8 Gbps for random lookup of a 4x4 size block, of 8-bit pixels under 100 MHz operation. Interleaving can be a cost-effective solution for fast pixel lookup in embedded computer vision.
  IEEE, English, International conference proceedings
  ISSN：1063-6919, Web of Science ID：WOS:000260371900203
• HIGH-S/N IMAGING OF A MOVING OBJECT USING A HIGH-FRAME-RATE CAMERA　　　　　　　　　　　　　　　
  Takashi Komuro; Yoshihiro Watanabe; Masatoshi Ishikawa; Tadakuni Narabu
  2008 15TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOLS 1-5, First page：517, Last page：520, 2008, [Reviewed]
  In this paper we propose a high-S/N imaging method involving combining many images captured with small blur using a video camera capable of high-frame-rate image capturing at 1000 frames/s. Use of a high-frame-rate camera makes the image change between frames small, enabling easy motion estimation, and makes it possible to use more light information, even when the exposure time is reduced to avoid blurring. To obtain a clear picture without misalignment due to motion parallax, it is necessary to determine both the motion and a depth map of the subject from noisy input images. We show results when applying the proposed algorithm to an image sequence captured by a high-frame-rate camera.
  IEEE, English, International conference proceedings
  ISSN：1522-4880, Web of Science ID：WOS:000265921400130
• Integration of Time-sequential Range Images for Reconstruction of a High-resolution 3D Shape　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  19TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION, VOLS 1-6, First page：408, Last page：411, 2008, [Reviewed]
  The recognition of dynamic scenes using 3D shapes could provide useful approaches for various applications. However the conventional 3D-shape sensing systems dedicated for such scenes have had problems in spatial resolution, though they have achieved high sampling rate in temporal domain. In order to solve this limits, we present a method that integrates time-sequential partial range images capturing moving targets to reconstruct a high-resolution range image. In the proposed method, multiple range images are set in the same coordinate system based on multi-frame simultaneous alignment. This paper also demonstrates the performance of the proposed method using some example rigid bodies.
  IEEE, English, International conference proceedings
  ISSN：1051-4651, Web of Science ID：WOS:000264729000100
• An image-moment sensor with variable-length pipeline structure　　　　　　　　　　　　　　　
  Atsushi Iwashita; Takashi Komuro; Masatoshi Ishikawa
  IEICE TRANSACTIONS ON ELECTRONICS, Volume：E90C, Number：10, First page：1876, Last page：1883, Oct. 2007, [Reviewed]
  A 128 x 128 pixel functional image sensor was implemented. The sensor was able to capture images at 1,000 frame/s and extract the sizes and positions of 10 objects/frame when clocked at 9 MHz. The size of each pixel was 18 mu m x 18 mu m and the fill factor was 28%. The chip, 3.24 mm x 3.48 mm in size, was implemented with a 0.35 mu m CMOS sensor process: the power consumption was 29.7 mW at 8 MHz.
  IEICE-INST ELECTRONICS INFORMATION COMMUNICATIONS ENG, English, Scientific journal
  DOI：https://doi.org/10.1093/ietele/e90-c.10.1876
  DOI ID：10.1093/ietele/e90-c.10.1876, ISSN：0916-8524, eISSN：1745-1353, Web of Science ID：WOS:000250095600005
• A Networked High-Speed Vision System for 1,000-fps Visual Feature Communication　　　　　　　　　　　　　　　
  Shingo Kagami; Shoichiro Saito; Takashi Komuro; Masatoshi Ishikawa
  First ACM/IEEE International Conference on Distributed Smart Cameras / Proceedings, First page：95, Last page：100, Sep. 2007, [Reviewed]
  English, International conference proceedings
  DOI：https://doi.org/10.1109/ICDSC.2007.4357511
  DOI ID：10.1109/ICDSC.2007.4357511
• A High-Speed and Compact Vision System Suitable for Wearable Man-machine Interfaces　　　　　　　　　　　　　　　
  Takashi Komuro; Bjoern Werkmann; Takashi Komai; Masatoshi Ishikawa; Shingo Kagami
  IAPR 10th Conference on Machine Vision Applications / Proceedings, First page：199, Last page：202, May 2007, [Reviewed]
  English, International conference proceedings
• Development of a 128×128 Pixel Image-Moment Sensor　　　　　　　　　　　　　　　
  IWASHITA Atsushi; KOMURO Takashi; ISHIKAWA Masatoshi
  The Journal of the Institute of Television Engineers of Japan, Volume：61, Number：3, First page：383, Last page：386, Mar. 2007
  We developed a smart image-sensor especially for extracting statistical values that contain areas,positions,orientations and shapes of the objects in an image.These values are called moments.The sensor has a small processing circuit in each pixel in order to achieve high resolution,but it outputs various information at real-time.We describe the implementation and performance evaluation of a 128 128 pixel image-moment sensor.
  The Institute of Image Information and Television Engineers, Japanese
  DOI：https://doi.org/10.3169/itej.61.383
  DOI ID：10.3169/itej.61.383, ISSN：1342-6907, CiNii Articles ID：110006854560, CiNii Books ID：AN10588970
• Real-time Three-dimensional Sensing for a Moving/Deforming Object using High-speed Vision for Numerous-point Analysis　　　　　　　　　　　　　　　
  WATANABE Yoshihiro; KOMURO Takashi; ISHIKAWA Masatoshi
  JRSJ, Volume：25, Number：6, First page：1005, Last page：1013, 2007, [Reviewed]
  This paper describes a real-time three-dimensional sensing system for applications based on feedback automation. The proposed system enables to observe a moving/deforming object at high frame rate and can acquire data in realtime. These are provided by three distinctive features, three-dimensional sensing by a single frame, high-frame-rate imaging and high-speed image processing. We also present some results of evaluation experiments. The experimental results show the advantages of our system compared with conventional approaches. Our system is expected to achieve improvements in a wide range of three-dimensional-sensing applications.
  The Robotics Society of Japan, Japanese
  DOI：https://doi.org/10.7210/jrsj.25.1005
  DOI ID：10.7210/jrsj.25.1005, ISSN：0289-1824, CiNii Articles ID：10019859749, CiNii Books ID：AN00141189
• 955-fps real-time shape measurement of a moving/deforming object using high-speed vision for numerous-point analysis　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  PROCEEDINGS OF THE 2007 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, VOLS 1-10, First page：3192, Last page：+, 2007, [Reviewed]
  This paper describes real-time shape measurement using a newly developed high-speed vision system. Our proposed measurement system can observe a moving/deforming object at high frame rate and can acquire data in real-time. This is realized by using two-dimensional pattern projection and a high-speed vision system with a massively parallel co-processor for numerous-point analysis. We detail our proposed shape measurement system and present some results of evaluation experiments. The experimental results show the advantages of our system compared with conventional approaches.
  IEEE, English, International conference proceedings
  DOI：https://doi.org/10.1109/ROBOT.2007.363965
  DOI ID：10.1109/ROBOT.2007.363965, ISSN：1050-4729, Web of Science ID：WOS:000250915303037
• A high-speed vision system for moment-based analysis of numerous objects　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  2007 IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING, VOLS 1-7, First page：2429, Last page：2432, 2007, [Reviewed]
  We describe a high-speed vision system for real-time applications, which is capable of processing visual information at a frame rate of 1 kfps, including both imaging and processing. Our system performs moment-based analysis of numerous objects. Moments are useful values providing information about geometric features and invariant features with respect to image-plane transformations. In addition, the simultaneous observation of numerous objects allows recognition of various complex phenomena. The proposed system achieves high-speed image processing by providing a dedicated massively parallel co-processor for moment extraction. The co-processor has a high-performance core based on a pixel-parallel and object-parallel calculation method. We constructed a prototype system and evaluated its performance. We present results obtained in actual operation.
  IEEE, English, International conference proceedings
  ISSN：1522-4880, Web of Science ID：WOS:000253487202006
• Design of a massively parallel vision processor based on Multi-SIMD architecture　　　　　　　　　　　　　　　
  Kota Yamaguchi; Yoshihiro Watanabe; Takashi Komuro; Masatoshi Ishikawa
  2007 IEEE INTERNATIONAL SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOLS 1-11, First page：3498, Last page：3501, 2007, [Reviewed]
  Increasing demands for robust image recognition systems require vision processors not only with enormous computational capacities but also with sufficient flexibility to handle highly complicated recognition tasks. We describe a multi-SIMD architecture and the design of a vision processor based on it for carrying out such difficult image recognition tasks. The proposed architecture consists of two SIMD parallel processing modules and a shared memory, allowing highly parallelized and flexible computation of complicated recognition tasks, which were difficult to process on a conventional massively parallel SIMD architecture. We designed a prototype vision processor for evaluation purposes and confirmed that the processor could be implemented in FPGA.
  IEEE, English, International conference proceedings
  ISSN：0271-4302, Web of Science ID：WOS:000251608404144
• A moment-based 3D object tracking algorithm for high-speed vision　　　　　　　　　　　　　　　
  Takashi Komuro; Masatoshi Ishikawa
  PROCEEDINGS OF THE 2007 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION, VOLS 1-10, First page：58, Last page：63, 2007, [Reviewed]
  In this paper we propose a method of realizing continuous tracking of a three-dimensional object by calculating moments of a translating and rotating object whose shape is known, either analytically or by using a table, and matching them with those of the input image. In simulation, the position and orientation is accurately recognized. Using a noise model and particle filter, we show that the position and orientation can be recognized even with noisy images.
  IEEE, English, International conference proceedings
  DOI：https://doi.org/10.1109/ROBOT.2007.363765
  DOI ID：10.1109/ROBOT.2007.363765, ISSN：1050-4729, Web of Science ID：WOS:000250915300010
• VCS-IV: A real-time vision system using a digital vision chip　　　　　　　　　　　　　　　
  S Kagami; T Komuro; Y Watanabe; M Ishikawa
  ELECTRONICS AND COMMUNICATIONS IN JAPAN PART II-ELECTRONICS, Volume：89, Number：6, First page：34, Last page：43, 2006, [Reviewed]
  A real-time visual processing system based on a vision chip that aggregates per pixel digital processing circuitry is presented. The system Supports the authors' latest vision chip and achieves high-speed, high-performance visual processing using an implementation that is small yet highly expandable and versatile. The main feature of the system is that the image capture function is programmable, with the operation of the internal pixel A-D conversion controlled from software at high time resolutions. The performance of the system is evaluated by application to basic visual processing and measurement systems, and examples of applications of the enhanced image capture function are presented. (C) 2006 Wiley Periodicals, Inc.
  SCRIPTA TECHNICA-JOHN WILEY & SONS, English, Scientific journal
  DOI：https://doi.org/10.1002/ecjb.20269
  DOI ID：10.1002/ecjb.20269, ISSN：8756-663X, Web of Science ID：WOS:000237889800005
• Parallel Extraction Architecture for Information of Numerous Particles in Real-Time Image Measurement　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Shingo Kagami; Masatoshi Ishikawa
  Journal of Robotics and Mechatronics, Volume：17, Number：4, First page：420, Last page：427, Aug. 2005, [Reviewed]
  English, Scientific journal
  CiNii Articles ID：10019859772
• Real-Time Shape Recognition Using a Pixel-Parallel Processor　　　　　　　　　　　　　　　
  Takashi Komuro; Yoshiki Senjo; Kiyohiro Sogen; Shingo Kagami; Masatoshi Ishikawa
  Journal of Robotics and Mechatronics, Volume：17, Number：4, First page：410, Last page：419, Aug. 2005, [Reviewed]
  English, Scientific journal
  CiNii Articles ID：10019381200
• A Pixel-Parallel Algorithm for Detecting and Tracking Fast-Moving Modulated Light Signals　　　　　　　　　　　　　　　
  Shingo Kagami; Masatsugu Shinmeimae; Takashi Komuro; Yoshihiro Watanabe; Masatoshi Ishikawa
  Journal of Robotics and Mechatronics, Volume：17, Number：4, First page：384, Last page：397, Aug. 2005, [Reviewed]
  English, Scientific journal
• Development of a Bit-level Compiler for Massively Parallel Vision Chips　　　　　　　　　　　　　　　
  Takashi Komuro; Shingo Kagami; Masatoshi Ishikawa; Yoshio Katayama
  7th IEEE International Workshop on Computer Architecture for Machine Perception / Proceedings, First page：204, Last page：209, Jul. 2005, [Reviewed]
  English, International conference proceedings
• Parallel Extraction Architecture for Image Moments of Numerous Objects　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Shingo Kagami; Masatoshi Ishikawa
  7th IEEE International Workshop on Computer Architecture for Machine Perception / Proceedings, First page：105, Last page：110, Jul. 2005, [Reviewed]
  English, International conference proceedings
• Multi-Target Tracking using a Vision Chip and its Applications to Real-Time Visual Measurements　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Shingo Kagami; Masatoshi Ishikawa
  Journal of Robotics and Mechatronics, Volume：17, Number：2, First page：121, Last page：129, Apr. 2005, [Reviewed]
  English, Scientific journal
• A High-speed Vision Chip and Robot Applications　　　　　　　　　　　　　　　
  Takashi Komuro; Shingo Kagami; Akio Namiki; Masatoshi Ishikawa
  2004 1st IEEE Technical Exhibition Based Conference on Robotics and Automation / Proceedings, First page：3, Last page：4, Nov. 2004, [Reviewed]
  English, International conference proceedings
• Real-time Visual Measurements using High-speed Vision　　　　　　　　　　　　　　　
  Yoshihiro Watanabe; Takashi Komuro; Shingo Kagami; Masatoshi Ishikawa
  SPIE Optics East 2004 / Proceedings, Oct. 2004, [Reviewed]
  English, International conference proceedings
  DOI：https://doi.org/10.1117/12.571049
  DOI ID：10.1117/12.571049
• A High-Speed Vision System with In-Pixel Programmable ADCs and PEs for Real-Time Visual Sensing　　　　　　　　　　　　　　　
  Shingo Kagami; Takashi Komuro; Masatoshi Ishikawa
  8th IEEE International Workshop on Advanced Motion Control / Proceedings, First page：439, Last page：443, Mar. 2004, [Reviewed]
  English, International conference proceedings
• A dynamically reconfigurable SIMD processor for a vision chip　　　　　　　　　　　　　　　
  T Komuro; S Kagami; M Ishikawa
  IEEE JOURNAL OF SOLID-STATE CIRCUITS, Volume：39, Number：1, First page：265, Last page：268, Jan. 2004, [Reviewed]
  Conventional SIMD image processors are very effective for early visual processing because of their parallelism. However, in performing more advanced processing, they exhibit some problems, such as poor performance in global operations and a tradeoff between flexibility of processing and the number. of pixels. This paper shows a new Architecture and sample algorithms of a vision chip that has the ability to reconfigure its hardware dynamically by chaining processing elements. A prototype chip with 64 x 64 pixels manufactured using the 0.35-mum CMOS process is also shown.
  IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, English, Scientific journal
  DOI：https://doi.org/10.1109/JSSC.2003.820876
  DOI ID：10.1109/JSSC.2003.820876, ISSN：0018-9200, eISSN：1558-173X, CiNii Articles ID：80016390913, Web of Science ID：WOS:000188205500031
• Improving the Sensitivity of a Vision Chip Using the Software A-D Conversion Method　　　　　　　　　　　　　　　
  Daisuke Takeuchi; Shingo Kagami; Takashi Komuro; Masatoshi Ishikawa
  IS&T/SPIE 16th Annual Symposium on Electronic Imaging Science and Technology / Sensors and Camera Systems for Scientific, Industrial, and Digital Photography Applications V, Proceedings of SPIE, Vol.5301, First page：138, Last page：148, Jan. 2004, [Reviewed]
  English, International conference proceedings
  DOI：https://doi.org/10.1117/12.526319
  DOI ID：10.1117/12.526319
• ビジョンチップのための動的再構成可能なSIMDプロセッサ(画像処理,画像パターン認識)　　　　　　　　　　　　　　　
  小室孝; 鏡慎吾; 石川正俊
  Volume：86, Number：11, First page：1575, Last page：1585, Oct. 2003, [Reviewed]
  従来の画像処理用SIMDプロセッサは,その並列性により主に初期視覚処理に対して強い威力を発揮するが,より高度な処理を行おうとした場合に,非局所演算が不得手であることや,画素数と汎用性の間にトレードオフが存在することなどの問題があった.本論文では,PE同士を結合させる機能をもたせ,ハードウェアを動的に再構成可能にした新しいビジョンチップのアーキテクチャとアルゴリズム例を示す.また,同アーキテクチャに基づき,0.35μmCMOSプロセスを用いて64×64画素を集積したチップを試作したので報告する.
  ISSN：0915-1923, CiNii Articles ID：110003170854
• A Software-Controlled Pixel-Level A-D Conversion Method for Digital Vision Chips　　　　　　　　　　　　　　　
  Shingo Kagami; Takashi Komuro; Masatoshi Ishikawa
  2003 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors / Proceedings, May 2003, [Reviewed]
  English, International conference proceedings
• A High Speed Digital Vision Chip with Multi-grained Parallel Processing Capability　　　　　　　　　　　　　　　
  Takashi Komuro; Shingo Kagami; Masatoshi Ishikawa
  2003 IEEE Workshop on Charge-Coupled Devices and Advanced Image Sensors / Proceedings, May 2003, [Reviewed]
  English, International conference proceedings
• Moment calculation method for digital vision chip　　　　　　　　　　　　　　　
  Idaku Ishii; Takashi Komuro; Masatoshi Ishikawa
  Systems and Computers in Japan, Volume：34, Number：1, First page：89, Last page：97, Jan. 2003, [Reviewed]
  Recently, an interesting general-purpose digital vision chip has been realized by integrating a photodetector (PD) and a processing element (PE) directly connected for each pixel on a single chip. This paper proposes bit plane (BP) feature decomposition as a realization of the idea of feature calculation that is suited to a massively parallel processing structure integrated on the digital vision chip. Moment calculation based on the method is discussed. An evaluation of the digital vision chip using the proposed calculation method is presented, and its effectiveness is demonstrated. © 2002 Wiley Periodicals, Inc.
  English, Scientific journal
  DOI：https://doi.org/10.1002/scj.1187
  DOI ID：10.1002/scj.1187, ISSN：0882-1666, SCOPUS ID：0037242487
• A digital vision chip specialized for high-speed target tracking　　　　　　　　　　　　　　　
  T Komuro; Ishii, I; M Ishikawa; A Yoshida
  IEEE TRANSACTIONS ON ELECTRON DEVICES, Volume：50, Number：1, First page：191, Last page：199, Jan. 2003, [Reviewed]
  This paper describes a new vision chip architecture for high-speed target tracking. The processing speed and the number of pixels are improved by hardware implementation of a special algorithm which utilizes a property of high-speed vision and introduction of bit-serial and cumulative summation circuits. As a result, 18 objects in a 128 x 128 image can be tracked in 1 ms. Based on the architecture, a prototype chip has been developed; 64 x 64 pixels are integrated in 7 nun square chip and the power consumption for obtaining the centroid of an object per every 1 ms is 112 mW. Some experiments are performed on the evaluation board which is developed for evaluation under the condition of actual operation. High-speed target tracking including multitarget tracking with collision and separation has successfully been achieved.
  IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC, English, Scientific journal
  DOI：https://doi.org/10.1109/TED.2002.807255
  DOI ID：10.1109/TED.2002.807255, ISSN：0018-9383, eISSN：1557-9646, CiNii Articles ID：80015872328, Web of Science ID：WOS:000181542400027
• An advanced digital vision chip and its system implementation　　　　　　　　　　　　　　　
  S Kagami; T Komuro; M Ishikawa
  SICE 2003 ANNUAL CONFERENCE, VOLS 1-3, First page：1568, Last page：1571, 2003, [Reviewed]
  We have been developing a digital vision chip, a CMOS image sensor that integrates a digital processing element and a photo detector directly connected in each pixel. In this paper, we report recent progress in our studies including an advanced vision chip based on a new PE architecture, and peripheral system implementation that support this new architecture.
  IEEE, English, International conference proceedings
  Web of Science ID：WOS:000222573800307
• ビジョンチップのためのマルチターゲットトラッキングとその応用(画像処理,画像パターン認識)　　　　　　　　　　　　　　　
  渡辺義浩; 小室孝; 鏡慎吾; 石川正俊
  Volume：86, Number：10, First page：1411, Last page：1419, 2003, [Reviewed]
  本論文では,ビジョンチップのための複数対象同時追跡を行うマルチターゲットトラッキングのアルゴリズムを提案する.本アルゴリズムは2分探索を用いたラベリング処理とSelf Window法によるトラッキング処理からなり,ビデオフレームレートで代表される従来の視覚システムにおけるものに比べて,高速性の点で優れている.更に,本アルゴリズムのアプリケーションとして,回転物体の回転軸,速度を計測する回転計測と画像上に出現する領域数を計測する個数カウントを提案する.提案するアプリケーションに関してビジョンチップによる評価を行い.100〜1000Hzといった高いフレームレートでの実現が可能であることを示す.
  ISSN：0915-1923, CiNii Articles ID：110003170837
• A New Architecture of Programmable Digital Vision Chip　　　　　　　　　　　　　　　
  Takashi Komuro; Shingo Kagami; Masatoshi Ishikawa
  2002 Symposium on VLSI Circuits / Digest of Technical Papers, First page：266, Last page：269, Jun. 2002, [Reviewed]
  English, International conference proceedings
• A Real-Time Visual Processing System using a General-Purpose Vision Chip　　　　　　　　　　　　　　　
  Shingo Kagami; Takashi Komuro; Idaku Ishii; Masatoshi Ishikawa
  2002 IEEE International Conference on Robotics and Automation / Proceedings, First page：1229, Last page：1234, May 2002, [Reviewed]
  English, International conference proceedings
• Digital vision chips and high-speed vision systems　　　　　　　　　　　　　　　
  M Ishikawa; T Komuro
  2001 SYMPOSIUM ON VLSI CIRCUITS, DIGEST OF TECHNICAL PAPERS, First page：1, Last page：4, 2001, [Reviewed]
  Conventional image processing has a critical limit of frame rate derived from serial transmission of the video signal. In order to overcome the limit, fully parallel processing architecture without scanning has been proposed. In this paper, vision chips with digital circuits and high speed application systems developed in our laboratory will be described.
  JAPAN SOCIETY APPLIED ELECTROMAGNETICS & MECHANICS, English, International conference proceedings
  Web of Science ID：WOS:000173132600001
• Device and System Development of General Purpose Digital Vision Chip　　　　　　　　　　　　　　　
  Takashi Komuro; Shingo Kagami; Idaku Ishii; Masatoshi Ishikawa
  Journal of Robotics and Mechatronics, Volume：12, Number：5, First page：515, Last page：520, Oct. 2000, [Reviewed]
  English, Scientific journal
• 1ms sensory-motor fusion system　　　　　　　　　　　　　　　
  M Ishikawa; T Komuro; A Namiki; Ishii, I
  ROBOTICS RESEARCH, First page：359, Last page：364, 2000, [Reviewed]
  Recently, there has been growing interest in sensory-motor integration for new behavior of intelligent robots. And the key component to this work is sensory information processing technology which is based on recent progress in the integration of electronic circuits, providing increased computing power at low cost. In this talk, a new type of vision chip which has a general purpose parallel processing array with photo detector in a single silicon chip will be discussed. The vision chip achieves 1 ms image processing, so mechanical systems can be controlled by using visual information with a 1 ms sampling rate. Additionally a 1 ms sensory motor fusion system, a new type of a hierarchical parallel sensory processing system, will be discussed. The system consists of integrated sensor modules and a parallel processing system providing for sensory feedback and novel performance. A demonstration of high speed grasping using visual and force feedback will be described.
  SPRINGER-VERLAG LONDON LTD, English, International conference proceedings
  Web of Science ID：WOS:000087440500044
• High speed target tracking vision chip　　　　　　　　　　　　　　　
  T Komuro; Ishii, I; M Ishikawa; A Yoshida
  5TH INTERNATIONAL WORKSHOP ON COMPUTER ARCHITECTURES FOR MACHINE PERCEPTION, PROCEEDINGS, First page：49, Last page：56, 2000, [Reviewed]
  This paper describes a new vision chip architecture for high speed target tracking. The system speed and pixel size improved by hardware implementation of a special algorithm which utilizes a property of high speed vision. Using an asynchronous and bit-serial propagation method, global moments of the image are calculated at high speed and with small circuits. Based on the new architecture a 64x64 pixel prototype chip has been developed.
  IEEE COMPUTER SOC, English, International conference proceedings
  Web of Science ID：WOS:000089941900006
• New generation of sensory information processing for intelligent systems - VLSI vision chip and sensor fusion system　　　　　　　　　　　　　　　
  M Ishikawa; T Komuro; A Namiki; Ishii, I
  ICEMI'99: FOURTH INTERNATIONAL CONFERENCE ON ELECTRONIC MEASUREMENT & INSTRUMENTS, VOLS 1 AND 2, CONFERENCE PROCEEDINGS, First page：1, Last page：6, 1999, [Reviewed]
  Recently, there are growing interests in applications of VLSI technology for sensory information processing based on the concept of "system on silicon". That means we can use fruitful computing power of compact VLSI at low cost. Therefore, the technology can open a new generation of intelligent systems. In this talk, a new type of vision chip which has general purpose parallel processing array with photo detector in a single silicon chip will be shown, The vision chip realize 1ms image processing, so mechanical systems can be controlled by using visual information with 1ms sampling rate. In addition, high speed grasping robot as a vision and force sensor fusion system will be shown, The system uses parallel DSP system for sensory feedback and novel performance can be implemented. A demonstration of the high speed grasping will be shown.
  CHINESE INSTITUTE ELECTRONICS, English, International conference proceedings
  Web of Science ID：WOS:000084980200001
• Vision chip architecture using general-purpose processing elements for 1ms vision system　　　　　　　　　　　　　　　
  T Komuro; Ishii, I; M Ishikawa
  CAMP'97 - FOURTH IEEE INTERNATIONAL WORKSHOP ON COMPUTER ARCHITECTURE FOR MACHINE PERCEPTION, PROCEEDINGS, First page：276, Last page：279, 1997, [Reviewed]
  This paper describes a vision chip architecture for high-speed vision systems that we propose. The chip has general-purpose processing elements (PEs) ina massively parallel architecture, with each PE directly connected to photo-detectors. Control programs allow various visual processing applications and algorithms to be implemented. A sampling rate of 1ms is enough to realize high-speed visual feedback for robot control. To integrate as many PEs as possible on a single chip a compact design is required, so we aim to create a very simple architecture. The sample design has been implemented into an FPGA chip; a full custom chip has also been designed and has been submitted for fabrication.
  I E E E, COMPUTER SOC PRESS, English, International conference proceedings
  Web of Science ID：WOS:A1997BJ77E00035

• A study of contactless vein authentication system using 3D images　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (B), 01 Apr. 2023 - 31 Mar. 2026
  Gunma University
  Grant amount(Total)：17290000, Direct funding：13300000, Indirect funding：3990000
  Grant number：23H01644
• 質感再現のための任意視点画像生成ネットワーク　　　　　　　　　　　　　　　
  01 Apr. 2022 - 31 Mar. 2025
  Grant amount(Total)：4160000, Direct funding：3200000, Indirect funding：960000
  Grant number：22K12088
• 非接触型指静脈認証の適用範囲拡大に関する研究　　　　　　　　　　　　　　　
  01 Apr. 2020 - 31 Mar. 2023
  Grant amount(Total)：16640000, Direct funding：12800000, Indirect funding：3840000
  Grant number：20H02386
• Computational Optical Imaging for Endscopic Surgery　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (S), 31 May 2017 - 31 Mar. 2022
  Nagahara Hajime, Osaka University
  Grant amount(Total)：150540000, Direct funding：115800000, Indirect funding：34740000
  It is difficult to measure the 3D shape of organs by Time-of-Flight technology since the human tissue is the translucent and closed environment inside the human body causing multi-path reflections. We developed the new camera to eliminate the effect of the multi-path of light reflections and scattering of the light. We also proposed the 3D reconstruction method from the camera. We developed endoscopic surgery supporting system which visualizes the inside of the organs and tracks the visualization with the deformation of the organ shape. Moreover, we applied some proposed methods to real surgery and evaluate the effectiveness of helping it.
  Grant number：17H06102
• Functional biomedical imaging with multi-aperture and multi-tap CMOS image sensors　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (B), 01 Apr. 2018 - 31 Mar. 2021
  Kagawa Keiichiro, Shizuoka University
  Grant amount(Total)：17550000, Direct funding：13500000, Indirect funding：4050000
  The architecture of the programmable space- and time-division multiplexed integrated camera and multi-aperture multi-tap CMOS image sensor as its core device for real-time imaging has been established. An 2x2-aperture four-tap CMOS image sensor that can perform the multi-band spatial frequency domain imaging (SFD) and multi-exposure laser speckle contrast blood flow imaging (MELSCI) concurrently has been designed and fabricated. Its operation was confirmed experimentally.
  Grant number：18H01497
• Study on simple and high-quality presentation of material appearance using augumented reality　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (C), 01 Apr. 2017 - 31 Mar. 2020
  Komuro Takashi, Saitama University
  Grant amount(Total)：4550000, Direct funding：3500000, Indirect funding：1050000
  In this study, we developed a new system that can reproduce the material appearance of various objects and that enables manual manipulation in 6 degrees of freedom (3 translation + 3 rotation). The result of evaluation experiments showed that matching user's visual and somatic senses improves the usability of the system. In addition, we developed a system to measure the three-dimensional shape and reflectance of a real object, which can be presented on the above system.
  Grant number：17K00266
• A study of contactless finger vein authentication systems for public facilities　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (B), 01 Apr. 2017 - 31 Mar. 2020
  Suzuki Hiroyuki, Tokyo Institute of Technology
  Grant amount(Total)：10790000, Direct funding：8300000, Indirect funding：2490000
  It is well known that biometric information can provide convenient and reliable user authentication. Recently, the application field of biometric authentication is expanding from small-scale service to wide-scale one such as big event venue. In these situations, high-speed calculation and smooth operation are required because of a large number of users. In order to apply finger vein authentication to above services, we propose a hand waving finger vein authentication system, in which a hand-waving movie is captured with a camera, and a biometric feature is obtained by synthesizing some finger vein patterns extracted from the movie. In our study, we developed a prototype system of the hand-waving finger vein authentication and evaluated the verification performance of the proposed system.
  Grant number：17H02036
• Research on object tracking and image restoration in the dark　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Challenging Exploratory Research, 2011 - 2013
  KOMURO Takashi, Saitama University
  Grant amount(Total)：3640000, Direct funding：2800000, Indirect funding：840000
  In this reseasrch, we proposed a method of realizing object tracking and image restoration in the dark in which target motion and a reference image are simultaneously estimated using a Bayesian framework. Deblurring is performed using the motion parameters estimated from the blurred images. As a result, we realized object tracking and image restoration from simulated video images with an SNR of up to -6 dB, and real video images captured in a dark environment of less than 0.01 lx illuminance at the subject surface. In addition, we examined the optimal frame rate for image restoration and we found that a higher frame rate was better under relatively little noise while a lower frame rate was better under much noise.
  Grant number：23650082
• Application of Vision Chip　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (S), 2009 - 2011
  ISHIKAWA Masatoshi; NAMIKI Akio; KOMURO Takashi; OKU Hiromasa; ALVARO Cassinelli; WATANABE Yoshihiro, The University of Tokyo
  Grant amount(Total)：110760000, Direct funding：85200000, Indirect funding：25560000
  We have developed a massively-parallel visual-information-processing chip called as Vision Chip. The developed chip has shown the effectiveness of the high-speed visual feedback at the speed of 1kHz which is largely increased in the speed compared with the conventional video-rate(30Hz). The fundamental technology for such high-speed visual feedback is now being established. This technology progress allows us to enter a new phase to challenge the application development for the real world. Based on this background, in this research project, we focus on two key applications, High-speed Visual Interface and High-speed Micro Visual Feedback. High-speed Visual Interface observes human motion in real-time by using high-speed vision and realizes new man-machine interaction systems. High-speed Micro Visual Feedback observes microorganism at high accuracy by controlling a microscope based on the feedback of the high-speed vision.
  Grant number：19100002
• Research on comfortable input interfaces using high speed vision　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Young Scientists (A), 2007 - 2009
  KOMURO Takashi, The University of Tokyo
  Grant amount(Total)：24700000, Direct funding：19000000, Indirect funding：5700000
  In this research, aiming to realize comfortable input interfaces for various information devices using high frame rate cameras which go over the video frame rate, and we built the following systems: 1)In-air mouse/keyboard for mobile devices, 2)Zooming touch panel, 3)Lip-reading interface, 4)Handy 3D scanner.
  Grant number：19680010
• 1MHzのビジュアルフィードバックを可能とするビジョンセンサの開発　　　　　　　　　　　　　　　
  2005 - 2006
  Grant amount(Total)：2700000, Direct funding：2700000
  Grant number：17656113
• Ultra-High-speed Sensory-Motor System Based on Distributed Network Architecture　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (S), 2002 - 2006
  ISHIKAWA Masatoshi; NAMIKI Akio; KOMURO Takashi; OKU Hiromasa, The University of Tokyo
  Grant amount(Total)：103870000, Direct funding：79900000, Indirect funding：23970000
  The purpose of this study is to develop a recognition-behavior integration system which can act in a flexible manner at high-speed in the real environment. Using basic technologies, such as sensory-motor integration based on hierarchical parallel architecture, ultra-high-speed recognition and motion, and distributed network processing architecture, we have developed a high-speed realtime processing system in which a large number of sensors and actuators are connected based on distributed network architecture.
  1.A high-speed vision network system with TCP/IP communication function.
  In order to track many targets which moves at high-speed in a wide area without occlusion, we have developed a distributed cooperative network system with multiple vision systems.
  2.A distributed sensor network information processing algorithm.
  We have proposed a sensor fusion processing method called DTKF (Delay-Tolerant Kalman Filter). DTKF is a method using Kalman Filter, and it has a good scalability against the increase of the number of sensors. The proposed method was verified by a numerical simulation of a target tracking task of high-speed vision sensor.
  3.A multi-high-speed visual feedback system.
  We have developed a multi-high-speed visual feedback system with multiple high-speed visions. By interpolating the lack of information using several vision systems, a 3D tracking of multiple targets which is robust against occlusion is achieved. Also, the improvement of a range of measurement and accuracy, and occlusion avoidance is achieved.
  4.Sensory-motor fusion based on hierarchical parallel processing architecture
  In order to connect a large number of sensor systems and robot systems, we have developed a parallel distributed processing system in which realtime processing systems, robot systems, and sensor systems are integrated. In the system three realtime processing systems made by dSpace Inc, are connected to each other using the CAN bus, and two robot arms, three robot hands, three high-speed vision systems (six camera heads), and other sensors (tactile sensors, force sensors, and so on) are connected with high-speed IO ports.
  Grant number：14102018
• Modeling and Model Learning in Very High Speed Visual Servo Systems　　　　　　　　　　　　　　　
  Japan Society for the Promotion of Science, Grants-in-Aid for Scientific Research, Grant-in-Aid for Scientific Research (A), 2002 - 2004
  HASHIMOTO Koichi; MORI Ryosuke; KOMURO Takashi
  Grant amount(Total)：50050000, Direct funding：38500000, Indirect funding：11550000
  Conventional manipulators are designed to control the end point position with high accuracy based on the joint angle measurement, and are not designed for high-speed control with vision sensors. Therefore, it is difficult to achieve an intellectual, high-speed task for enabling the performance of high-speed vision. And thus the speed-up of the whole visual feedback system highly depends on the speed-up of the manipulation system. We think that introducing high-speed actuator is not enough and it is important to understand the characteristics of the image processing system and the active vision, the hand system and also the design of the control system. In this sense, to achieve the super-high-speed visual servo system, we have to develop the model that describes the dynamics of all elements composing the system ; and the development of the parameter identification technique ; and the model learning algorithm. The following research are done this year :
  1.Visual servo by high-speed hand system : A high-speed multi finger hand system was developed. An algorithm to catch a falling ball and a falling column stably was proposed, and the performance of the super-high-speed visual servo was evaluated by experiments.
  2.Microorganism tracking system : A system that can stably observe a swimming paramecium was developed. The position of the pool where a lot of paramecia swim was controlled by a XY stage.
  3.Expansion of stability area of visual servo : In this research, the method of expanding the stability area was examined in consideration of the restriction with limited camera view.
  4.Composition and robustness in visual servo system : In this research, it was pointed out that the robustness of the system changes greatly by combining different structures and algorithms.
  5.Ball capture and lifting task : Ball catching and lifting experiments have been done.
  Grant number：14205034
• ビジュアルサーボの大域的安定化と特徴量空間経路計画　　　　　　　　　　　　　　　
  2001 - 2002
  Grant amount(Total)：2200000, Direct funding：2200000
  Grant number：13875080