Pursuing Efficient Visual Recognition to Match Human Capabilities
Exploring neural-network-based CV research focused on efficient perception for vehicles and robots.
The field of Computer Vision (CV) covers a broad range of topics, but my primary research focuses on processing visual information using neural networks for applications such as Autonomous Driving (AD) and Advanced Driver-Assistance Systems (ADAS). More specifically, I work on recognizing the surrounding environment by processing vision signals—typically from cameras—with the aim of realizing safer and more comfortable driving assistance and autonomous driving.
Because vision signals involve large volumes of data, efficient processing is critically important. I became interested in event-based cameras, which detect motion in a manner similar to biological eyes. Although this approach lies somewhat outside the mainstream of CV research, I felt that if we could process these change-based signals effectively, it might be possible to achieve a dramatic improvement in efficiency. This led me to begin research on neural networks specialized for such signal processing. In addition, as part of our broader efforts toward efficient recognition, I also work on neural network compression. By applying pruning techniques at the bit level with high precision, we developed Bit-Pruning, which suppresses accuracy degradation. We have also developed compression techniques such as structured feature-map sparsification, which reduces computational cost by repurposing sparse arithmetic units built into general-purpose GPUs provided by companies such as NVIDIA. These technologies have been presented at top-tier machine learning conferences, including ICLR, as part of our external outreach.
My research focuses on robot vision, with primary applications in manufacturing environments—particularly the automation of visual inspection processes. The goal is to automate visual inspection tasks that are currently performed by human visual checks, using various types of cameras. These include RGB cameras like those found in smartphones, spectral cameras that capture light across fine-grained frequency bands, cameras that visualize object states through polarization, and multi-camera systems that simultaneously capture images from multiple viewpoints. By leveraging this diversity of cameras, we aim to achieve highly accurate and efficient visual inspection.
Another important theme is vision for controlling robots. For example, in production sites where robots are used to automate screw-fastening tasks, conventional robots operate under the assumption that screws are placed at predefined positions. Making such systems work requires meticulous adjustment by on-site personnel and considerable time. If CV can serve as the robot’s “eyes,” however, robots that previously had to work blindly based on precise positioning can instead recognize the positions of screws and screw holes on their own and perform installation autonomously. This technology is applicable not only to assembly tasks but also to emerging global efforts such as the disassembly of end-of-life vehicles for recycling. Since each vehicle differs slightly in condition, tasks must be performed while visually assessing where components are located. This is a technology that can also contribute to the circular economy.
Combining with Large Language Models Raises Challenges in Accuracy, Speed, and Power Efficiency
Integrating CV and language models while balancing performance, speed, and energy constraints.
Recently, multimodal generative AI—including CV—has been attracting significant attention. In AD/ADAS applications, for example, there is growing interest in generating dangerous driving scenarios from text that are difficult to collect in the real world and using them as training data, as well as applications in which driving instructions are given verbally during autonomous driving. These areas require large-scale resources, both in terms of researchers and computational infrastructure, making closer collaboration with DENSO headquarters and universities increasingly important.
While combining CV with Large Language Models (LLMs) has enabled many new possibilities, there are still many cases where such technologies cannot yet be used in real-world production environments. For example, technologies are emerging that can analyze an image and answer a question such as, “Is there anything abnormal here?” However, in actual manufacturing lines, the detection speed often cannot keep up with the speed at which parts move along the line. Deciding how to use cutting-edge technologies and how to translate them into practical solutions is crucial. The key challenge lies in balancing accuracy, speed, and cost. Improving and optimizing these factors is what we see as the next step in our research.
Generative AI is becoming increasingly capable, with expanding modalities and functionalities. Yet tasks that humans can process with minimal energy often require enormous amounts of power—sometimes tens of thousands of kilowatts—when handled by AI systems. As things stand, such technologies cannot be implemented on edge devices such as those used in factories or vehicles. This is why reducing power consumption is an extremely important topic. Exploring how to make high-performance AI more compact and capable of running at the edge is both an intellectually engaging challenge and an area where we can directly contribute to DENSO’s business. Solving these difficult problems through advanced research is, in my view, the very reason ITLAB exists.
Motivation Comes from Both Academic Achievement and Social Implementation
Research motivation driven by both academic recognition and real-world deployment.
Researchers at ITLAB are motivated to conduct research for different reasons. In my case, I am happy when a paper is accepted at a top-tier conference, and I am equally happy when a technology I developed is incorporated into a product and released into the world. Of course, achieving better experimental results than before in our daily work is also a strong source of motivation.
Personally, I am only interested in having someone actually use the results of my research. One of the strengths of ITLAB is that DENSO provides a clear path toward implementation as the closest “exit.” Several of my technologies have already been implemented in services offered by DENSO or its subsidiaries. While implementing automotive technologies can take time because manufacturing lines must be modified, there are still ample opportunities for deployment—particularly in new business areas and production-related technologies like mine.
One defining feature of ITLAB, for better or worse, is the very high degree of freedom in setting research themes. Even researchers who have just joined the company can independently define their own topics, as long as they can articulate a story about how their work will contribute to DENSO’s business. Conversely, this environment may be challenging for those who prefer to work on predefined topics. ITLAB is well suited to individuals who have a strong desire to pursue their own ideas and who want to start research from a clear vision or concept.
Looking ahead, I would like to place greater emphasis on research themes closer to robotics. Until now, ITLAB has primarily focused on software, while DENSO has focused on hardware. However, I feel that important challenges lie precisely at the boundary between these domains. Although ITLAB is a place where software researchers gather, and it is not always easy to establish hardware-oriented topics such as robotics as research themes, I am nevertheless determined to step into this area and pursue it further.
As I mentioned earlier, the freedom to define research themes is one of ITLAB’s strengths. At the same time, themes defined by a single individual tend to remain relatively small in scale, making it difficult to tackle truly large research challenges. By teaming up with DENSO, partner companies, and students from Institute of Science Tokyo, I hope to take on a major challenge—realizing efficient AI—that would likely be impossible for me to achieve alone.
Join Our Research at DENSO ITLAB
Where engineering curiosity
meets real-world impact
