Reimagining High Performance Compute Infrastructure: A New Era of Compute
Your Title Goes Here
Your content goes here. Edit or remove this text inline or in the module Content settings. You can also style every aspect of this content in the module Design settings and even apply custom CSS to this text in the module Advanced settings.
Notes
As AI workloads continue to explode across enterprise infrastructure, IT leaders are grappling with the limitations of traditional homogeneous compute models. In this episode of The Hitchhiker’s Guide to IT, host Michelle Dawn Mooney sits down with Justyn Horner, CEO of Ionix, to explore how heterogeneous compute is reshaping enterprise infrastructure—and why organizations need more flexible strategies to drive performance without complete overhauls.
With over twenty years of experience in product development, AI/ML systems, and hardware optimization across major enterprises, Justyn brings a unique perspective from his unconventional journey that began in the music industry. He breaks down how organizations can leverage diverse processor types, overcome vendor lock-in challenges, and build composable infrastructure that adapts as AI demands evolve.
Key topics include:
- What heterogeneous compute means—and why mixing CPUs, GPUs, ASICs, and FPGAs is becoming critical for AI workloads
- The role of composable infrastructure in future-proofing against rapid hardware evolution
- How to orchestrate workloads across diverse processors using unified programming approaches
- Why IT leaders should focus on power efficiency and cost optimization when planning AI infrastructure
- Real-world success stories showing dramatic performance improvements and cost reductions
- The importance of starting with low-risk pilot projects to build confidence in heterogeneous environments
Whether you’re a CTO planning your AI infrastructure roadmap or an IT operations manager evaluating compute strategies, this episode provides practical insights into the technical realities, workforce implications, and strategic opportunities of heterogeneous compute for enterprise AI initiatives.
Transcript
Welcome to the Hitchhiker’s Guide to IT, brought to you by Device42. On this show, we explore the ins and outs of modern IT management and the infinite expanse of its universe.
So buckle up and get ready to explore the ever changing landscape of modern IT management.
Hello, everyone, and welcome to the Hitchhiker’s Guide to IT. I’m your host, Michelle Dawn Mooney. Today’s episode explores a topic that is rapidly reshaping enterprise infrastructure, heterogeneous compute. As traditional compute models struggle to keep up with AI and data intensive workloads, IT leaders need new strategies to drive performance and efficiency without ripping and replacing everything.
So joining me is Justyn Horner, CEO of Ionix. With more than twenty years of experience in product development, AI, ML systems, and hardware optimization, Justyn has helped some of the world’s largest enterprises adopt smarter, more flexible compute strategies. Justyn, thank you so much for joining me. Welcome to the show.
It’s great to be here, Michelle. Thank you.
Yeah. Loving what we’re gonna talk about today because we have some good stuff to hit on today. But before we do that, can you give us a brief bio if you can, please?
Sure. Yeah.
My first industry was actually in music. So I was a touring artist and learned to write code when I launched a record label as a way to recruit bands. So, did that first, and that was how I first got into software development.
And then went on to do that and, worked in several large data centers, including those at Walmart where I grew up in Bentonville, Arkansas. And I made my way out to, Las Vegas, And, I’ve been building technology in the city at large scale for most of the casinos in the region for for many years, built many different, artificial intelligence platforms.
This is also my seventh startup, a Techstars alum, and I just can’t get it out of my system. I love building love building products.
That’s fantastic. And I have to say this will go down as probably the most surprising bio when it comes to what I was expecting or what I’ve heard before.
Never in a million years would I have thought that it would start in the music industry and you were a traveling musician. I wish we had more time to talk about that, but maybe at the end if we can get through all the questions. But that is so awesome. So loving that aspect too. So starting at a high level, what’s driving the shift from traditional homogeneous infrastructure towards heterogeneous compute infrastructure?
Yeah. No. There’s a a lot of factors influencing it. One of the biggest ones that we find from our customers is the concerns about vendor lock. So a lot of the big chipmakers right now are locking in their customers and essentially building monolithic ecosystems where you have to go all in on one vendor.
And that’s creating a lot of risk in the market. And so we we’re seeing a big shift towards trying to create a lot more flexibility so that organizations can adapt as new chips, new data comes into the system. They need to process it differently. So giving people the flexibility to use the chips that they need to solve their problems, a big a big factor.
The other one is the market is beginning to understand that different chips do different things. You need to run different kinds of math. There’s different trade offs, everywhere. So NVIDIA went from Hopper to Blackwell.
We see AMD with their Instinct series, for example. Intel has Gaudi. These all have different trade offs in what these chips can do, and organizations are becoming more aware that certain problems require different chips. And, vendor lock is creating it and making it even more difficult and less efficient.
So we built our platform to help them orchestrate the right chips at the right time.
So let’s dive a little deeper here because we’re talking about the shift towards heterogeneous compute. But what exactly is heterogeneous compute, and why is it becoming so critical for enterprises, especially those focused on AI and high performance workloads?
Yeah. The traditional approach is artificial intelligence has really, of course, exploded even over the last five, six years.
The traditional approach is to use a CPU and a GPU to solve that problem.
And what we’re finding is there’s lots of other chipsets available, not just within GPUs themselves, but we’re seeing ASICs, we’re seeing FPGAs, and other technologies, NPUs, DPUs. There’s all these different kinds of chips to solve the problems.
So being able to put the right chip into the mix to solve that problem is heterogeneous compute. So it’s combining these chips to solve the problem as opposed to a traditional CPU GPU only approach.
Let me ask you about putting things into practice because it’s like the saying goes, the proof is in the pudding. So what are some real world benefits organizations are seeing from hybrid compute environments?
Right. So, typically, especially with AI, there’s two kind of legs. There’s the training side, and then there’s the inference side. So training is when you use your data to build a new model.
Once you’ve built that model, then you need to deploy it and let your either your employees, your, or your customers use that model that you trained. So those are very different use cases. So on the training side, what we do is we enable with different chip sets, data preprocessing so that you’re not putting all that workload into the GPU, for example. We also have challenges that we help solve around precision.
So depending on the kind of math that they need to operate on these chips, all these chips have trade offs. And so as we’re helping our customers understand, depending on the kinds of data that they’re processing, the models that they’re building, they might need a chip that does much higher precision. So we can help them integrate that right into their work flow and make much more sophisticated models. Then when you move that model over for inference, there’s a six FPGAs, far more efficient, chipsets that enable them to run those at scale.
They’re less expensive. They’re a lot easier to manage as long as there is a platform that helps orchestrate those.
I love hearing success stories, and I’m pretty sure you probably have seen a few in your time. So can you walk us through a use case where a heterogeneous setup drastically improved performance or maybe reduced costs?
Yeah. So one that we’ve been working on with the proof of concept right now, it’s gone it was a big success. We got all all green lights on it last Friday.
Was a use case where we were doing data preprocessing on ASICs.
So in this case, the customer had very large amounts of data that was, duplicated across many different datasets. So instead of trying to throw all of that into a CPU and a GPU where I think approximately forty percent of the workload was just in data cleaning, we were able to do all that data cleaning on the ASICs. And then by the time the workload got into GPUs, which are much higher power and a lot more expensive, they were able to use a lot fewer GPUs to that workload. So that drove down their cost. It was also far less expensive.
A six tend to range between anywhere from a thousand to four thousand dollars compared to even lower in GPUs or in the forty thousand dollar range. So you see massive cost benefit, there.
From a CapEx standpoint, it was a lot less expensive for them to build this. But then these ASICs that we use in this case were three hundred watts compared to seven hundred and fifty on the GPUs. So significant power savings as well. What we’re finding is a bit of a segue here is that folks are using all the available power that they have. So, if they have a megawatt available, they will use all of that for the compute, and we’re able to leverage the right chip at the right time to give them a lot more compute for the power that they need.
You’ve mentioned GPUs, ASICs, FPGAs. How do you orchestrate workloads effectively across such a diverse set of processors?
That was extremely painful, just to be candid. Because each of these have different compilers. They have different ways that you write software and deploy kernels to them. And so we had to build a compiler process, and we use the language called Rust.
So, this gets really sophisticated quickly, but I think your your, your listeners will appreciate it. There’s a concept called intermediate representations in the software development world where we can go from Rust down to a different language and then compile that to the chip that it’s gonna be operating on. So you write your code once and you can deploy it to any chipset. So we set out to do that a few years ago and discovered that we were able to do that successfully, but we had no way to network them up effectively.
So that became our next few years we’ve been working on is putting together a network mesh, a topology that allows us to put all of these chips together in a single, in a single rack, or across multiple racks disaggregated. So and this is where the composability becomes so powerful is that you can put all of these chips in a single rack. You can put them across different racks. We are still able to, connect them up, with extremely low latency.
We talked about in the beginning of the show the change. It’s just happening so rapidly, and there can be a bit of trepidation for companies that need to be flexible to embrace some of these changes. So for IT leaders listening who are hesitant, how can they begin exploring heterogeneous compute without overhauling everything?
Well, I would encourage folks if they’ve been building, AI products internally and particularly, they could they should start and stay there. So we built our ecosystem to run on Python so they don’t have to go through a major lift and shift on their code development. So I would encourage them to, start at that stage of their company where they’re already building, deploying on Python to GPUs, and then we can help them lift and shift that hardware into our into our stack that allow them to orchestrate more chips as they become more comfortable. And so this is a a a safe, sane way.
They probably already own their own chips. So they would we would just need to bring in our network fabric to allow them to connect those up. And at that point, it’s, day zero. They can still deploy with what they’ve been doing.
There’s not a major change if you’re just going CPU to GPU. And then we can start layering in more chips to help them solve those problems. A lot of what we’re finding, for example, is we’ve got folks that are very comfortable with NVIDIA’s CUDA ecosystem, and they want to leverage the higher precision available with AMD, for example.
So we can help them make that transition in a very painless way so that they can start testing with those chips, find the sweet spot, and then even blend and blur them together. That’s what we’re finding some of our customers are moving towards, using AMD for one part of the stack, NVIDIA for the other, maybe AMD for training, NVIDIA for inference. So we give them that flexibility without the major shift where they have to start in Rust.
That was one of our hard lessons we learned earlier this year when we came to market was immediate concerns that what we’ve been building in Python, what we wrote all of our own CUDA files. So we spent the last couple of months making that transition and being able to stay on Python and then giving you the tools to transition as you’re ready.
Yeah. I think it could be overwhelming for a lot of people looking at the big picture. And, you know, baby steps will get you there, not necessarily as fast as some people might like. But what are some low risk pilot projects that can help teams gain confidence through this process?
Now one of the benefits that our system has, for example, most GPUs are deployed on a server in eight at a time. So we’re able to do up to forty eight, accelerators on a single server. What that lets them do is, be able to scale up. And so that’s one of the first tests that I help folks go through is let’s transition your eight GPU clusters to a forty eight GPU cluster. And we can do that, whether they’re on SXM, OAM, or PCIe.
Those are different form factors that GPUs can be plugged into a server. And so, whatever form factor they’re currently using, we can help them transition to our network fabric very efficiently. Then that’s a really great way for them to immediately have access to a lot more GPUs in a much simpler ecosystem.
And that’s just a big win right there for them. It’s it’s fewer servers that they have to operate, fewer parts that break. It’s easier to network them all up in a single ecosystem. If they’re running Python, Python then is aware of all forty eight accelerators available to it. It’s a very easy way for them to go from what they’re currently doing to a much more sophisticated composable infrastructure that then we can start layering heterogeneity into that.
As a follow-up to that, what role do orchestration and workload intelligence play in making these environments manageable?
It’s critical, because you are working especially if you go heterogeneous, now you’ve got different chips that are all talking different languages. So the, the orchestration layer that we built enables that to happen not just within a rack, but across racks, even across data centers. So orchestrating those workloads is critical. A big part of our road map is the ability to dynamically provision resources that are available or underutilized on a particular workload.
So the longer it runs, it learns, and then is able to leverage those chips more effectively as it goes. So these are the kinds of things that we enable. You can’t do that today because it’s just a big, homogeneous compute cluster. When you’ve got heterogeneity, it enables the system to be aware of underutilized chips and be able to leverage those to either improve speed or improve power consumption, whatever the the customer is looking to do.
It’s amazing to see how AI is really just transforming all sectors of the industry, but especially in tech. So as AI becomes more embedded in enterprise infrastructure, what new demands will that place on compute strategies?
Well, ultimately, all AI comes from the data you have. So, the ability to leverage your data, and I think you’re gonna see more strains on, the organization’s ETL data science teams being able to leverage the data, aggregate the data, annotate the data for training. We’re gonna see a lot more emphasis on that as organizations learn that they have something special that they can build a model and solve problems for their customers. So we’re gonna see a lot of strain in there.
We’re also gonna see, of course, on the compute side. So, I work a lot with different legislators and coalitions that are actively influencing state and local governments to make sure they have the power consumption available to, data centers or on prem solutions. So we’re seeing a lot of demand on the grid in certain communities. So one of the benefits of a composable infrastructure that we’ve built is that you can start small and expand as the grid, stabilizes or more, power is available on that grid.
So giving organizations the ability to scale up and down as needed is a a really powerful piece of composable infrastructure.
And, of course, we have the heterogeneous compute in there, which further improves power consumption when you’re leveraging the right chip at the right time.
Of course, those chips are a effectively.
So those are two parts of the demand side that we’re seeing on the organization with the data science teams and within the communities as you need to scale up your compute to build those products and run them, you’re gonna need data centers and power. And so we’re working on both sides of that within communities and within the organizations.
And the one thing we know for sure is that everything will continue to be ever changing because you never know what tomorrow or the next day or the next week, month, etcetera, will show. So what trends are you watching closely as we move into the second half of twenty twenty five and even beyond?
I’ll take a more philosophical point of view on this one. We’re seeing a lot of organizations that successfully embed artificial intelligence into their workflows, especially as a Genentech AI becomes more prominent.
We’re seeing significant, let’s call it workforce efficiencies that are putting demand on fewer people. For example, internally, we ran a study where our engineers work forty two times more effectively when partnered with artificial intelligence and agents.
That has led us to having to hire a lot less, and it’s also creating challenges with, junior entry level engineers having a place.
So we tend to hire super seniors, one percenters, to build our software stack. And we’re actively working to create, you know, apprentices apprenticeships, internships that will allow junior engineers to come in and begin to work in our ecosystem.
So we’re seeing that trend across other industries, whether it’s legal, medical, you name it. There are knowledge workers now that when partnered with AI and AgenTek AI are able to work far more effectively.
And I think we’re we’re watching closely how those trends are gonna impact communities. So we’re actively engaged in workforce, groups within different states to help them with a plan for how to educate people so that they have they know how to use these tools and so that when they can move into an organization, they’re already familiar with Agintiq AI, how to leverage them. Every organization is gonna be different. The tools that they choose and build are gonna be very different.
So there’s gonna be a learning curve, but it’s gonna be absolutely critical that citizens, especially knowledge workers, know how to leverage these tools and that organizations embed those tools into, the fabric of how they work. And that is gonna have a huge impact. So that’s a trend we’re watching closely as communities adapt and evolve, to the workforce shifting tremendously, and it’s gonna accelerate. And organizations that don’t embed AI have serious risk of their competitors who do.
Because those that do are gonna be able to accelerate and build and, solve customer problems much more faster than those that don’t.
So besides the philosophical side of things, what other trends are you seeing on the more technical side?
Two trends really emerge right now. One is that as folks are embracing this heterogeneous compute approach, they’re able to build entirely new models that didn’t exist and don’t exist today because they have access to different kinds of chips that can do different kinds of math. The other one that we are building really quickly towards right now is data storage. So traditional data storage is more about high bandwidth, how many users can extract the files that they have stored.
High performance compute data storage is very different. So we built an extremely low latency data storage solution that allows for, during AI training in particular, data movement, which is critical for keeping those processors running efficiently, and they’re not sitting around waiting for the data to load. So we built our own flavor of that just so we could support our customers as those as those challenges were emerging. This kind of network fabric and a composability enabled us to build that, in a way that we haven’t seen anybody else in the market addressing.
So any final advice for infrastructure teams trying to future proof their strategy for AI and data intensive workloads?
Yeah. I’d strongly recommend composable infrastructure. It’s gonna give you more flexibility. It’s gonna future proof. So in our case, for example, new chips come out. You don’t have to replace the entire CapEx of, of your racks and your network fabric and all these other things, software. You can literally install the chip, into the rack and keep going.
That’s gonna help with, adapting as you learn about your own data and the models you need to build and customer demands. You can adjust your chipsets to solve those problems more effectively.
So composable infrastructure is a really key part of that. And where we layer into that is, you know, the, heterogeneity that allows the right chips at the right time. That’s where you can see more efficiencies, lower cost, to deploy. And so we think that this is a a critical part for AI and infrastructure teams to understand well is how how composable infrastructure can help future proof. And, a lot of this has to do with, like, the network topology, how you network these chips up, how you, orchestrate them across, nodes of compute and scaling those up. So that’s gonna be, I think, a lot of the big learning over the next couple years is is how to build those systems out, maintain, and operate them. And what we’re finding is that organizations that shift to this, you know, for example, again, not having eight, servers in Iraq, you only have one.
There’s a lot less, downtime. There’s a lot less service of, servicing of parts that break because parts break. The manufacturing side of this world is still growing and learning. So we see a lot of failure rates on big chips.
Making sure you have a plan and the composability makes servicing those kinds of chips, a lot easier for everybody. So there’s a lot of benefits to composability besides just ease of use. It’s that first learning curve of, like, well, how do we put this thing together? And we’ve got solutions for that.
I have to ask this before we let you go. Of course, I wanna touch on this before I ask my last question, but where can people go if they want more resources and they are saying, you know what? Everything that you’re saying sounds like something that we should be thinking about. So where can you send them, Justyn?
Sure. Ionix h p c dot com. It’s your website.
We are actively engaging with proofs of concept for for organizations. So, I mean, you’re happy to have a discovery call and go straight into, purchasing, but what we found is folks wanna see it work for them. So we’re engaging with proofs of concept now with organizations.
I recommend that we work with your most advanced AI team internally, that the folks that are gonna understand the technology most quickly, they’re also gonna know what problems they’re trying to solve. And so we like to engage with that group first and, help them understand how we can, solve that problem with them. And then from there, we’ll help them put together, the right infrastructure to help solve that problem at scale.
Okay. So now I have to ask, going back to the beginning of the show, starting in the music business and working as a traveling artist, what does the CEO of IONIX have on their playlist?
On my playlist is wild.
So classically trained pianist, I’m gonna give a shout out to one of my favorite artists. His name is Avery Bright, a a wonderful, virtuoso violinist and composer.
So he’s somebody I’ve been listening to a lot lately, and he happens to be the artist on our video when you go to ionix h p c dot com and you watch our video.
He’s he’s such a favorite artist of mine that I reached out to him and asked if I could use one of his pieces on our video, and he agreed. So, I love working with I love working with the smaller artists. If you ever come to our office, it’s filled with art from our local, artists that in residence that we have. And so, so engaging with the artist community is a big goes all the way back to my roots as an artist myself.
Yeah. Now can we find Justyn Horner anywhere? Can we listen to Justyn?
Oh my goodness. If you find something, send it my way. It’s been a long time. It’s been a long time.
I still play. I was a pianist, was my, was my instrument, my primary instrument. So I I got a grand grand piano at my house. And if the kids are out of the house, and they’re not gonna yell at me for playing, I’ll sit down and, and and tap out of tune every now then.
To be continued, we will see if we can dig up anything on Justyn Horner. Justyn Horner, CEO, Ionix. Thank you so much, Justyn, for your time. Gave us some very valuable information, some good insight towards the future.
Things are changing. And if we don’t catch up, we might be falling out of the of the race of everybody else who is going to be going farther, and and we’re not gonna be able to even compete. So appreciate your time today. Thank you so much.
Michelle, always a pleasure.
And I wanna thank you for tuning in and listening to the Hitchhiker’s Guide to IT. A big thank you again to Justyn Horner for joining us, breaking down how heterogeneous compute can help IT leaders get more performance, efficiency, flexibility without a complete infrastructure overhaul. And if you enjoy today’s episode, be sure to subscribe wherever you get your podcast, and don’t forget to explore more episodes focused on the tools and strategies shaping the future of IT. I’m your host, Michelle Dawn Mooney. Thanks again for joining us. We hope to connect with you on another podcast soon.