Behind every AI breakthrough is a chip doing the heavy lifting. But as artificial intelligence becomes more powerful, the cost of computing and its growing energy demands are raising important questions. semXai, a startup spun out of IIT Hyderabad, is looking to address these challenges with indigenous, energy-efficient AI chips. CE speaks to co-founder Dinesh YVS about building technology for India’s AI future
Excerpts
Tell us about semXai and the idea behind starting the company.
The idea behind the startup and the technology started just after COVID, while I was exploring my career options. I was always keen to build chips and work on the indigenisation of technology, especially in semiconductors. These were a few thoughts I had after finishing my master’s degree. I thought about starting a company sometime in 2022. But credibility matters in this field, especially considering the investor ecosystem in India. I was looking for the right co-founder, and after searching for a long time, I realised a professor would be a better co-founder for me, someone who matched my frequency and could complement most of my ideas. I spoke to a professor at IIT Hyderabad, and we joined hands. I enrolled for a PhD and started building the technology with him. We raised nearly $4 million in grants, including support from the Indian Defence Ministry, the Ministry of Electronics, Qualcomm, AMD, and other companies that helped us build this technology. With the grant support, we eventually spun out of IIT Hyderabad last year and started semXai.
What gap did you identify in the market, and what made you believe semXai could address it differently?
AI is not a single wave. There are different sets of waves coming ahead. There were companies that came in around 2017 or 2018 and eventually couldn’t sustain in 2020 or 2021, post-COVID or during COVID. Again, new companies were born around 2022, saying they were addressing certain problems within AI. As AI evolves, AI hardware and AI chips evolve. The global leaders are doing well with generic chips, especially CPUs or GPUs. But there are startups building AI-native chips that complement the generic chips built by NVIDIA, AMD, or Intel, and that is where we are positioned. We are trying to make maximum use of the next wave of AI that is going to come around 2027–28, with robots, autonomous vehicles, drones, and many things working autonomously. We are trying to address that market. Most of our innovation is futuristic, and we want to ensure that we benefit as much as possible from the next wave of AI.
What were some of the biggest challenges you encountered while building the company, from developing your own chip architecture to assembling the right team and resources?
I think I can address this problem in two ways. First, the talent pool is very important for building AI chips. Since I am at IIT Hyderabad and am a student myself, master’s and undergraduate students work along with me. At any given point of time, 15 to 20 members work on different problem statements under me. These could involve software, compilers, AI design, or fabrication-related activities. There are many things happening, and students are doing master’s and undergraduate theses. While doing this, I identified something interesting: people are increasingly interested in building things where they have the freedom to build, rather than going to corporates. In this way, I managed to bring together many students, including PhD scholars, master’s students, and undergraduates. I must admit that being at IIT means I get access to a strong talent pool every year or every semester. Identifying the right people who can work in this space matters, regardless of their CGPA or the courses they have taken. Even a chemical engineer could be fantastic at AI modelling, and that is happening in my company. Irrespective of branches or departments, I am in the right place to identify the right talent pool and have the people who can do this job.
The other problem is the ecosystem for building chips. People struggle to get tool support for chip design. There are also challenges in working with Taiwan Semiconductor Manufacturing Company or GlobalFoundries, especially for fabrication and packaging. Fortunately, we decided not to start the company until we had figured out the ecosystem and completed a couple of chip fabrications. That helped us a lot. We completed two chip iterations even before starting the company, and the entire ecosystem was in place, from design tools and IPs to fabrication, packaging, and PCB support. The entire semiconductor-chip value chain was in place even before we planned the company. This is helping us because our focus is now on raising investments rather than building collaborations, since we already have those collaborations.
The startup focuses on sovereign and energy-efficient AI chips designed for applications ranging from data centres to drones, robotics, and autonomous vehicles. How did you arrive at this approach, and what went into deciding the applications you wanted to build for?
I truly believe in two things. The first is the market. Wherever the market is big, you have to address it so you can run the company in the best way possible. The market decides the vision of the company. We can develop disruptive technology, but it is the market we are trying to address, and introducing a disruption can create huge revenues.
The other important point we considered is sustainability in terms of running AI. Right now, gigawatts or even multiple gigawatts of power are being generated to supply data centres. Do we really need such a large electricity demand, or can we reduce it intelligently while still addressing the problems? We started thinking about the issues at the grassroots level: are they inside the chips, or is it the way AI is being developed that causes the problem? We understood that there are problems to address across different layers and abstractions of compute or applications. We started with a bottom-up approach and began building chips, retrofitting software layers wherever possible, and achieving energy savings. This hardware and software co-design approach is producing fascinating results when we benchmark it. The market driving the company’s vision and the focus on sustainability and reducing the electricity required to run AI were the primary factors that eventually decided our product lines.
What milestones are you working towards for the future, and how do you see the company contributing to India’s ambition to build a more self-reliant AI and semiconductor ecosystem?
We feel that India does not really have sovereign AI chips. We have demand for technologies such as ChatGPT and Gemini to be integrated into different sectors, and unfortunately, we rely on generic chips such as CPUs or GPUs. What if we had an AI-native chip on which these applications could run without requiring the generic ones, while saving electricity and water? As an Indian company, a second-order benefit would be much cheaper compute costs. This would ensure that we are building our own AI as well. The ripple effect would not just be that we are building chips; we would also be allowing developers to access cheaper compute so they can build better AI going forward.
In terms of milestones, we are addressing chatbots right now, and then we will address more robots and physical AI, which is the next wave coming up. For the next four to five years, we will continue building better chips and ensuring that the cost of computing complex applications or complex AI is much cheaper.
