Pragnya Nalla: Research Spotlight

1. Tell us about the findings of your recent work entitled “HISIM: A Tool for Fast Design Exploration of 2.5D/3D/3.5D Heterogenous Intergration for AI Computing” and how it supports the goals of CoCoSys. 

As AI models grow increasingly complex, deploying them on monolithic chips poses significant manufacturing challenges, including yield limitations. To address these issues, 2.5D and 3D integration architectures have been proposed. However, as AI models scale, the number of system-level design parameters expands exponentially, creating a pressing need for advanced tools capable of comprehensive design exploration. To fill this gap, we developed HISIM, a fast, accurate, and end-to-end simulation tool. HISIM employs analytical models calibrated with real chip data, RTL, and SPICE results. The tool supports a wide array of inputs, including computing cores like in-memory computing (IMC), systolic arrays, CPUs, AI algorithms spanning CNNs, LLMs, and Vision Transformers, as well as interconnection models such as TSVs, wire models, and Network-on-Chip (NoC) or Network-on-Package (NoP) components, including NoC routers, AIBs, and DDR modules.

HISIM is open source and available at GitHub, with the goal of empowering designers to make informed decisions and drive innovation in 2.5D/3D system simulations. Architectural and system-level modeling frameworks like HISIM are critical for enabling hardware and algorithm co-design, as envisioned in the CoCoSys Center’s Theme II: Hardware-Algorithm Co-design. This work contributes to Project ID 3131.08: System Evaluation and Benchmarking, aiming to develop tools for design space exploration of both individual hardware accelerators and systems comprising multiple accelerators. HISIM represents a significant step toward scalable, efficient, and collaborative design innovation in the AI hardware ecosystem. HISIM is the result of teamwork by Zhenyu Wang (Intern, TSMC), Pragnya Nalla (myself), Jingbo Sun (Intel), Ziyao Yang among others. For the full list of contributors, visit our GitHub page.

2. How do your research findings push the boundaries of what we currently know or can do in the field of chiplet simulation?

Traditional tools for AI design space exploration (AI DSE) are often limited to 2D analyses or focus on specific device or subsystem-level simulations. These approaches, while precise, are highly time-consuming due to their reliance on SPICE or cycle-accurate methodologies, making them less practical for comprehensive AI design exploration.In contrast, our innovative tool HISIM leverages analytical models, calibrated with real chip data and RTL or simulation tools, to enable 2.5D, 3D and 3.5D AI DSE with simulation times in the millisecond range. This breakthrough achieves unprecedented scalability, speed, and accuracy. Compared to existing AI benchmarking tools, it demonstrates a performance improvement of 10^4x to 10^6x in speed. With the ability to explore thousands of design configurations, our tool delivers insights into performance, power, and area (PPA) differences of up to 10x, enabling designers to optimize critical parameters like TSV pitch and the number of NoC links. Moreover, it facilitates early-stage design decisions by revealing their impacts on PPA, thermal performance and costs, significantly accelerating the path to optimal design solutions.

3. What are some real-world applications or examples of your research that people might encounter in their daily lives?

Imagine a company developing its next-generation chip, transitioning to 2.5D and 3D architectures to support cutting-edge AI applications while lowering manufacturing costs. The chip architect faces the daunting task of making decisions involving hundreds of design parameters. The number of possible design permutations grows exponentially, creating significant challenges. With current tools, the architect cannot perform a full system-level simulation to assess the impact of all permutations within a single platform. Moreover, traditional simulations are time-intensive, leading to prolonged development cycles and skyrocketing costs. Our open-source solution, HISIM, addresses these challenges by providing a seamless, scalable, and efficient platform. HISIM allows designers to gain actionable insights quickly, enabling faster decision-making and shorter product development timelines. This acceleration not only reduces development costs but also minimizes manufacturing expenses, ultimately driving down the cost of the final product while bringing innovations to market faster.

4. What inspired you to pursue this research, and why do you think it is important?

My journey into advancing energy-efficient, cost-effective designs for AI applications stems from firsthand experience. While working at an innovative medtech startup focused on using AI for contactless sleep diagnosis, I observed the critical challenges posed by hardware limitations. Despite the ingenuity of the software solutions, the device’s overall performance, battery efficiency, and costs were ultimately constrained by the computing chip. These limitations directly impacted the affordability and accessibility of the technology, highlighting the urgent need for better AI hardware solutions. This experience underscored a larger, pressing issue: as AI models continue to grow in complexity and impact, their adoption remains hindered by skyrocketing hardware costs and inefficiencies. These barriers prevent AI from being truly democratized, limiting its potential to benefit society universally. Motivated to address this gap, I pursued research aimed at making AI hardware more sustainable, accessible, and efficient. As part of this effort, we developed HISIM, an open-source tool designed to empower designers with faster decision-making capabilities. By enabling efficient exploration of AI design spaces, HISIM accelerates innovation in AI hardware, bringing us closer to a future where AI technology is both affordable and efficient.