Key Trends from 2024: Cell-based Architecture, DORA & SPACE, LLM & SLM, Cloud Databases and Portals

Transcript

Daniel Bryant: Hello and welcome to the InfoQ podcast. My name is Daniel Bryant. I'm the news manager here at InfoQ and my day job, I currently work in platform engineering at Syntasso. Today we have a treat for you as I've managed to assemble several of the InfoQ podcast hosts to review the year in software technology, techniques and people practices. We'll introduce ourselves in just a moment and then we'll dive straight into our review of architecture, culture and methods, AI and data engineering, and the cloud and DevOps. There is of course no surprises that AI features heavily in this discussion, but we have tried to approach this from all angles and we provided plenty of other non-AI insights too. Great to see you all again. Let's start with a quick round of intro, shall we? Thomas, do you want to go first?

Thomas Betts: Yes, sure. I don't think I've done my full introduction in a while on the podcast. My day job, I'm an application architect at Blackbaud, the number one software provider for social impact. I do a lot of stuff for InfoQ, and this year a lot of stuff for QCon. So I'm lead editor for architecture design, co-host of the podcast obviously, I was a co-chair of QCon San Francisco this year and a track host for QCon London, so that kind of rounds out what I've been up to. Next up, Shane.

Shane Hastie: Thanks, Thomas. Really great to be here with my colleagues again. Yes, Shane Hastie, lead editor for culture and methods. My day job, they call me the global delivery lead for Skills Development Group. Written a couple of books deep into the people and culture elements of things, the highlights for this year. Unfortunately, I didn't get to any of the QCons, I'm sad about that, and hopefully next year we'll be there in person. But some of the really amazing guests we've had on the Engineering Culture podcast this year have been, we'll talk a bit about it later on, but that's probably been some of my highlights. Srini.

Srini Penchikala: Thanks, Shane. Hello everyone. I am Srini Penchikala, my day job is, I work as an application architect, but for InfoQ and QCon, I serve as the lead editor for data engineering and AI and ML community at InfoQ. I also co-host a podcast in the same space, and I am serving as the programming committee member for QCon London 2025 conference, which I'm really looking forward to.

Real quick, 2024 has been a marquee year for AI technologies and we are starting to see the next phase of AI adoption. We'll talk more about that in the podcast and also I hosted a AI/ML Trends podcast report back in September, so that will be the biggest reference that I'll be going back to a few times in this podcast. Definitely there are a lot to look forward to. I am looking forward to sharing in today's podcast the technologies and trends that we should be hyped about and also what is all the hype that we should be staying away from. Next, Renato.

Renato Losio: Hi everyone. My name is Renato Losio. I'm an Italian cloud architect living in Germany. For InfoQ, I'm actually working in the Cloud queue, I'm an editor. And definitely my highlight of the year has to be being the chair of the first edition of the InfoQ Dev Summit in Munich. Back to you, Daniel.

Daniel Bryant: Fantastic. Yes, I've already enjoyed the Dev Summit. I was lucky enough to go to the InfoQ Dev Summit in Boston. I've worked in Boston for a number of years or worked with a Boston-based company and the content there was fantastic, staff plus in particular sticks in my mind. I know we're going to dive into that, but I also did the platform engineering track at QCon London this year, which is fantastic. Great Crossing paths with yourself, Srini, I think I've met almost everyone this year. Maybe not yourself, Shane, this year. Actually, I can't remember exactly when, but it's always good to meet in person and the QCons and the Dev Summits are perfect for that kind of stuff, and I always learn a ton of stuff, which we'll give the listeners a hint at tonight, right?

Did our software delivery trends and technology predictions from 2023 come true? [04:10]

So today as we're recording, so I just want to do a quick look back on last year. Every year we record these podcasts, we always say we want to look back and say, "Hey, did our predictions come true?" And when we pulled out this time for last year, we said, "2024, could the use of AI within software delivery becoming more seamless and increasing divide between organizations and people adopting AI and those that don't, and a shift towards composability and improved abstractions in the continuous delivery space". So I think we actually did pretty well this year, right? I'm definitely seeing a whole lot of AI, as you hinted at, Srini, coming in there and as you say, every year, Thomas, the AI overlords have not stolen our jobs yet. So not completely in terms of software engineering.

Thomas Betts: I think we're all still employed, and I'm surprised that quote in there about the separation, I think that's true. We're seeing the companies that are doing the innovator early adopter are still doing new things. I think the companies that are more late majority are like, "We want to use it", but they're not quite sure how yet. I don't know if, Srini, you have any more insight into how people are adopting AI?

Srini Penchikala: Yes, that's very true, Thomas. Yep, definitely AI is here, I guess, but again, it's still evolving, right? So I definitely see some companies are going full-fledged, some companies are still waiting for it to happen. So as they say that the new trend, and I will talk more about this later in the podcast, the agentic AI. The AI agents that cannot only generate and predict the content and insights, they can also take actions. So the agentic AI is a big deal. So as they say, the AI agents are coming, so whether they'll overtake our jobs or not, that's to be seen. But speaking of the last year's predictions, we had talked about the shift towards AI adoption, right? Adoption has been a lot more this year, but I think we still have some areas where I thought we would be further ahead and we are not. So it's still evolving.

Shane Hastie: Yes. I see lots and lots of large organizations that are not software product companies putting their toes in and bumping against privacy, security, ethics and not sure how to go forward knowing that they absolutely need to, and often that governance frame slowing things down as they're exploring, "Well, okay, what does this really mean for us?" And a lot of conservatism in that space.

Daniel Bryant: It's really funny, Shane, compared to what Renato and I are seeing. So I went to the Google Cloud Summit in London and I only heard AI, AI, AI, AI. If you listen to the hype, and I think, Renato, you covered re:Invent for us recently. I think if you sort listen to the hype, you believe everyone, even the more late adopters are covering AI, Renato, right?

Renato Losio: Yes, I mean, just to give a order of magnitude, I don't know if they changed the number during the conference, but at a conference like the re:Invent, there were over 800 session about AI/ML. By comparison, there were just about 200 about architecture and even less about serverless. So that gives a bit of direction where the conference was going.

Surprisingly, the itself was not so generative AI focused. They tried to make it difference, probably go back to that later on, but I find it interesting what happened in the last year in the space of AI cloud. But I don't take responsibility for the prediction of last year because I was not there. But I have to admit that I love to start with looking back at the prediction actually, when I see tech prediction for 2025, actually, I tend to go to the bottom of the article, use it as a reference to the article of the year before because I to see a year later what people predicted if it still does. I love to go back to those topics.

What are the key trends in architecture and design? [08:08]

Daniel Bryant: That's awesome, Renato, and you will get the privilege this time at the end to talk about your predictions, right? So we'll hold you to it next year. So enjoy it for the moment, but I think that's a perfect segue, you mentioned serverless there. Architecture and design is one of our sort of marquee topics. To be fair all the things we're going to talk about today are marquee topics, but we often look to the InfoQ through an architect lens. And Thomas, you sort of run the show for us there. I know you've got a bunch of interesting topics that you wanted to talk about. I think the first one was around cell-based architectures, things like that.

Thomas Betts: Yes, so this was I think something we highlighted in the A&D trends report back in April I think it was, and we ended up having an e-mag come out of it and had a lot of different topics. Some of those were from various articles or presentations at QCons. And just this happens in architecture that we have these ideas that have been around for a while, but we didn't have the technology to make it easy to implement. And then it becomes easier and then people start adopting it. So the idea of putting all of the necessary components in one cell, and so you minimize the blast radius.

And if one goes down, the next cell isn't affected, and how do you control it and how do you maintain it? Just like any microservices, distributor architecture, there's extra complexity involved, but it's becoming easier to manage, and if you need the benefits, then it's worth the trade-offs. You're willing to take on the extra overhead of managing these things. So like I said, that e-mag is full of a lot of different insights, different viewpoints. Again, architects always looking at different viewpoints and ways to look at a problem, so I like that it gives a lot of different perspectives.

Daniel Bryant: We've got to named check Rafal [Gancarz] there on that one just quickly, and Rafal did a fantastic job on that.

Thomas Betts: Yes, thanks for calling out Rafal. He did fantastic doing that. The other thing that I remember talking about a few people at QCon London this year, and I think QCon London last year as well, was the green software trends. So fantastic book just came out in April. I think the book signing was the release party at QCon London.

Daniel Bryant: Oh, yes, it was Anne Currie and team, Yes, fantastic.

Thomas Betts: Anne Currie, Sara and Sarah. Sara Bergman and Sarah Hsu were all there together and they actually credited InfoQ with being the force that made the book happen because that was how they were able to all get together and collaborate. So that book about Building Green Software, Adrian Cockroft has talked about this. He's kind of championing it from here's how you do it in the cloud, going back to serverless, he advocates make things serverless, make them small, only running when you need it. That kind of philosophy, I think we're going to start seeing architects have to start thinking about that more and more. It's going to become more important.

The point that I love about, and Sara, I had a great presentation on this, is that it just makes more efficient software. It makes better software, it makes more sustainable, more maintainable software, all the other abilities we look for, if you build with a green mindset, you get all those other benefits. So if you just say, "Oh, we need to make it green, we need to reduce our carbon footprint, and nobody really cares about that". Well, all the other things you do care about, they come along for the ride. So start thinking about that way. How do you only run your software when you need to? How do you only write the code you need? So there's a lot of ideas in there and I think we're going to start seeing more of that hopefully in the next year. That's definitely one of my 2025 predictions.

Renato Losio: I think you really raised a good point about software in the sense that it's seen as well as a proxy to other things like cost for example. I think if you go to your CFO and you say, "We are trying to make our cloud deployment greener". Probably he won't care that much. Even if outside the company you might sell that message that you're generating less CO2 realities. Often it's really a proxy on cost optimization. When you talk about serverless run just what you need or whatever, choose region that are more carbon effective often are as well the cheapest data center because they're more efficient. So it's an interesting approach to looking through the lens of green and not just cost or architecture.

Thomas Betts: Yes, I know Anne has talked a lot about how one of the only measures we have sometimes is what's our bill? What's the AWS bill that we get or Azure bill? And if that's the only measure you have, then use it, that's better than nothing and it is actually still a pretty good proxy. The cloud providers are getting better at saying, "Here's how that energy is produced". But you have to dig for it. I think we're going to start seeing that become more of a first class, "Here's your bill and then here's the carbon footprint of this because you ran it in this data center, which is in US East and it's running on coal versus somewhere in Europe that's on nuclear". Right? So I think that's going to be interesting to see if we can get to those metrics and people say, "Oh, we can run this somewhere else because there's a better carbon efficiency and we save a lot of money by doing it".

Srini Penchikala: All of that is important, I agree with you both the green software and the sustainable computing is a very good time to talk about it in the AI context because as we all know, the large language models that are the main part of GenAI, they do require a lot of processing power. So we have to be more conscious about how much are we really spending and how much value we are getting, right? So between the value of these solutions and what are we really spending in terms of money and energy resources and the environment, right? So how about you Shane? What do you think about green?

Shane Hastie: Green software, ethics and sustainability have been a drum I have wanted and have beaten for the last three years, and it's been great to see more and more the ability to have those hard conversations. And the challenging within organizations where as software engineers, as the development community, we can actually start to ask for, "Hey, we want to do it this way". And now as Thomas so smoothly says, we can actually use money as a good excuse, and that helps because without showing the measurable benefits, we're going to struggle.

Thomas Betts: And Srini, you brought up the AI component and yes, the AI carbon footprint is enormous. Somebody will say it's the next Bitcoin, it's just spending a lot of money, but hopefully it's producing value. The other aspect I thought was interesting, and this was a presentation at QCon San Francisco, was how GitHub Copilot serves 400 million requests per day, and it got into the details of how you actually have to implement AI solutions. So GitHub Copilot, two things. There's GitHub Copilot Chat, that's where you open a window. You ask a prompt, it responds, right? It's like ChatGPT with code, but GitHub Copilot the autocomplete, it's kind of remarkable because it has to listen for you to stop typing and then suggest the next thing.

And so all of the complications that go underneath that that I hadn't considered, so they had to create this whole proxy and it's hanging up HTTP two requests, and if you just use native things like engine X after a hundred disconnects, it just drops the connection entirely and so you've lost it. There's all these low level details, and I think when we get to see AI become more standard as parts of big software, more companies that have these distributed systems are going to run into some of these problems. Maybe not to GitHub Copilot scale, but there's probably these unintended things that are going to show up in our architecture that we haven't even thought of yet. I think that's going to be real interesting to see how AI creates the next level of architectural challenges.

Srini Penchikala: Also, maybe just to add to that, Thomas, maybe the AI can also solve some of those challenges, right? We talk about AI hallucinations and bias, but can AI also help minimize the environmental hallucinations and the ethical biases

Daniel Bryant: Said like a true proponent, Srini, the cause of and solution to most of the world's problems, Ai, right? I love it.

Thomas Betts: Yes, I think we're going to see how are we going to use AI as an architect? Can I use it in my day job? Can it help me design systems? Can it help me solve problems? I use it as the rubber duck. If I don't have someone else that I can get on a call and chat with and discuss a problem, I'll open up ChatGPT and just start a conversation and say, "Hey, I'm trying to come up with this. What are some of the trade-offs I should consider?" I haven't gone so far as to say, "Solve this problem". The hallucination may be completely invalid or it may be that out of the box thinking that you just hadn't thought of yet, it's going to sound valid either way. You still have to prove it out and make it work.

What are the key trends in culture and methods? [16:22]

So I think the other part of being an architect, I talked about using AI to do your job, but I think the architectural process has been a big discussion this year. All of the staff plus talks at QCons are always interesting. I think we have good technical feedback, but people love, I personally love the how do I do my job? How do I get better at my job, how do I level up? So we've seen some of that in decentralizing decision making. I talked to Dan Fike and Shawna Martell about that. They gave a presentation, they wrote up an article based on that presentation. And, Shane, I can't remember if you talked to them as well or you talked to somebody else about how to do a better job. How do you level up your staff plus, how do you become a staff plus or principal engineer?

Shane Hastie: Yes, I've had a number of people on the podcast this year talking about staff plus and growth both in single track and dual track career paths. The charity majors still going, the pendulum back and forward. When do you make your choices? How do you make your choices? Shameless plug, I had Daniel as a guest talking about people process for great developer experience, technical excellence, weaving that into the way that we work. So all of these things leveraging AI, challenging the importance of the human aspect, that critical thinking, and Thomas, you made the point, the hallucination is there.

Well, one of the most important human skills is to recognize the hallucination and not go down that path to utilize the generative AI and other tools at your fingertips most effectively. Platform engineering, the myth still of the 10 Xx engineer, but with great platform engineering, what we do is we make others 10 times better and there's a lot of the, I want to say same old people stuff still coming up because fundamentally the people haven't changed.

Daniel Bryant: That's a good point.

Shane Hastie: Human beings, we don't evolve as rapidly as software tools, digging into product mastery. Really interesting conversation with Gojko Adzic about observability at the customer level and bringing those customer metrics right to the fore. So Yes, all of the DORA metrics and all of these others still really important. I had a couple of conversations this year where we've spoken about DORA's great, and it is becoming a target in and of itself for many organizations, and that doesn't help if you don't think about all of the people factors that go with it.

Thomas Betts: There's a law that once you have a named goal, it stops being a useful metric, I'm botching the quote entirely. But it's like once you have that, it was useful to measure these things to know how well you're doing, but then people see it as that's the thing I have to do, as opposed to, "No, just get better and we'll observe you getting better".

Daniel Bryant: Goodhart’s law, Thomas.

Thomas Betts: Yep, thank you.

Shane Hastie: And W. Edwards Deming, if you give a manager a numerical target, they will meet it even if they have to destroy the organization to do so.

Thomas Betts: You mentioned the DORA metrics. Why I loved the QCon keynote Lizzie Matusov gave was talking about the different metrics you can measure for psychological safety and how to measure team success. And that's how you can say, are these teams being productive? And there's different survey tools out there and there's different ways to collect this data, but I think she focused on if people feel they're able to speak up and raise issues and have open conversations, that more than anything else makes the team more successful because then they're able to deal with these challenges.

And I think that keynote went over really well with the QCon audience, that people understood like, "Oh, I can relate to that, I can make my teams better". You might not be able to use all the new AI stuff, but you can go back and say, "Here's how I can try and get my teams to start talking to each other better". Like you said, Shane, the humans haven't evolved that fast, software has.

Daniel Bryant: That's a great quote. On that notion, are you seeing more use of other frameworks as well? Because I'm with you. In my platform engineering day job, I see DORA, every C-level person I speak to knows what DORA is, and for better or worse, people are optimizing for DORA, but I'm also seeing SPACE from [Dr] Nicole Forsgren, et al. I'm seeing DevEx from the DX folks, a few other things. And I mean space can be a bit complicated because there's like five things, the S, the P, the A, the C and the E. but I think if you pick the right things out of it, you can focus more on the people to your point and the productivity, well, and the happiness as well, right?

Shane Hastie: Yes, we could almost go back to the Spotify metrics. The Spotify team culture metrics have been around for a long time and what I'm seeing is reinventions of those over and over again. And it's fundamentally about how do we create that psychologically safe environment where we can have challenging conversations with a foundation of trust and respect. And that applies in technical teams of course, but it applies across teams and across organizations and the happiness metrics and there are multiple of those out there as well.

Ebecony gave us some good conversations about creating a joyous environment and also protecting mental health. Burnout is endemic in our industry at the moment, and I think it's not just in software engineering, it's across organizations, but mental health and burnout is something we're still not doing a good job at. And we really need to be upping our organizational gain in that space.

Renato Losio: I think it's been actually a very bad year in this sense as well with what you mentioned, Shane, about a manager that reached the goal, might destroy a team, make me think that this year one of the key aspects has been all the return to office mandate, regardless if it's good or not for the company, has been a key element of a large company, became like the new trend.

Shane Hastie: There's cartoons of people coming into the office and then sitting on Zoom calls because it's real. The return to office mandates, and this is a strong personal opinion, they're almost all have nothing to do with value for the organization, and they're all about really bad managers being scared. And I'm sure I've just made a lot of managers very unhappy.

Daniel Bryant: Hey, people join this podcast for opinions, Shane, keep them coming.

Thomas Betts: So many times come across the Conway's law and all different variations of this. I think Ruth Malan is the one who said that if you have an organization that's in conflict, the organization structure and conflict of the architecture, the organization is going to win. And I'm wondering what the return to office mandates, how is that going to affect the architecture? I made the quote, I think it was last year, the comment about the COVID corollary to Conway's law that the teams that can't work effectively, once we all went remote, they weren't able to produce good distributed systems because they couldn't communicate.

Well, now we're in that world. Everyone has adapted to that. I think we're seeing more companies are doing distributed systems and the teams don't sit next to each other. They have to form their own little bubbles in their virtual groups because they're not sitting at the desk next to each other. If we make people go back to the office, but we don't get the benefits of them having that shared office space, then what is that going to do to the software? I don't know if I have an answer to that, but it seems like it's not going to have a good impact if you're doing it for the wrong reasons.

Srini Penchikala: Maybe I can bring a cheesy analogy to this, right? We started this conversation with the serverless architecture where you don't need to run the software systems and servers all the time. They can be ideal when you don't need them. I think that should apply to us as humans also, right? I read this quote on LinkedIn this morning, I really like this, it says, "Almost everything will work again if you unplug it for a few minutes, including you". So we as humans, we need to unplug once in a while to avoid the burnout. I mean, that's the only way we can be productive when we need to work, if we have to take that break or time off.

Daniel Bryant: Yes. Plus one to that, Srini. Changing subjects a little bit. But Shane, I know you wanted to touch on some of the darker human sides of tech too.

Shane Hastie: Yes, I will go there. Cybercrime, the use of DeepFakes, generative AI. I had Eric O'Neill on and the episode is titled, Spies, Lies in Cybercrime and he brings the perspective from an FBI agent and there are some really interesting applications of technology in that space and they're very, very scary. Personally, I was caught by a scammer this year and they were the social engineering, it worked.

Fortunately, my bank's fraud detection system is fantastic and they caught it and I didn't lose any money, but it was a very, very scary period while we were trying to figure that out. And for me, and in Eric's conversations, it's almost always the social engineering piece that breaks the barrier. Once you've broken the barrier, then the technology comes into play. But he tells a story of a deepfake video in real time that was the chief financial officer of a large global company. So very perturbing, somewhat scary. And from a technical technologist perspective, how do we make our systems more robust and more aware? So again, leveraging the AI tools is one of the ways. So the potential for these tools is still huge.

What are the key trends in AI and data engineering? [27:01]

Daniel Bryant: Perfect segue, Srini, into your space. It's slightly scary, but well-founded grounding there, Shane. But Yes, I think it's almost a dual-use technology in terms of for better or worse, right? And, Srini, love to hear your take on what's going on in your world in the AI space.

Srini Penchikala: Yes, thanks, Daniel. Thanks, Shane, that's a really good segue. This is probably a repeat of one of the old quotes, Uncle Ben from Spider-Man movie, right? "With power comes responsibility". I know we kind of keep hearing that, but with powerful AI technologies, how to come the responsible AI technologies. So in the AI space, again, there are a lot of things happening. I encourage our listeners to definitely watch the 2024 trends report we published back in September. We go into a lot of these trends in detail, but just to highlight a couple of things in the short time we have in today's podcast, obviously the language models are going at a faster pace than ever, large language models, it seems like there's no end to them. Every day you see a new LLM popping out and the Hugging Face website, you go there, you see a lot of LLMs available for different use cases.

So it's almost like you need an LLM, large language model to get a handle on these LLMs. But one thing I am definitely curious about and also seeing the trend this year are what are called small language models. So these are definitely a lot smaller in terms of size and the data sets compared to large language models. But they are excellent for a lot of the use cases where again, talking about green software, you don't want to expend a lot of computing resources, but you can still get similar accuracy and benefits also. So these SLMs are getting a lot of attention. Microsoft has something called Phi-3, Google Gemma, there is the GPT Mini I think. So there are a lot of these small language models are definitely adding that extra dimension to the generative AI. And also these language models are enabling the AI modeling and execution on the mobile devices like phones, tablets, and IoT devices.

Now you can run these language models on a smaller phone or a tablet or laptop without having to send all the data to the cloud, which could have some data privacy issues and also obviously the cloud computing resource and cost. So this is one of the trends I'm watching, definitely keep an eye on this. And the other trend is obviously the one I mentioned earlier called agentic AI, agent-based AI technologies, right? So this is where I think we are going to the next level of AI adoption where we have the traditional AI that's been used for a long time for predicting the results. And then we have GenAI, which started a few years ago with the GPT and the ChatGPT announcements. So generative AI not only does the prediction, but it also generates the content and the insights, right? It goes to the next level. So I think the agents are going to go one more step further and they can not only generate the content or insights, but also they can act on those insights with or without supervision of humans.

So there are a lot of tasks that we can think of that we don't need humans to be in the middle of the process. We can have the agents act on those. And also one of the interesting use cases I heard recently is a multi-agent workflow application where each of the agents take the output from the previous agent as the input and they perform their own task. But doing so, they're actually also giving the feedback to the previous agent on the hallucinations and the quality of the output so the previous agent can pick a different model and rerun the task, right?

So they go through these iterations to improve the quality of the output and minimize the hallucinations and biases. So these multi-agent workflows are definitely going to be a big thing next year. So that's one other area that I'm seeing. Also, recently, just a couple of days ago, Google actually announced Google Gemini 2.0. The interesting thing about this article is Google's CEO, Sundar Pichai, he actually wrote a note, so he was kind of the co-author of this article.

So it was a big deal from that standpoint. And they talk about how the agentic AI is going to be impactful in the oral AI adoption and how these agentic AI models like Google Gemini 2.0 and other models will help with not only the content generation and insights, but also actually acting on those insights and performing the tasks. Real quickly on a couple of other topics, the RAG, we talked about this last year, retrieval augmented generation. So I'm seeing a couple of specialized areas in this.

One is multimodel RAG where you can use the RAG techniques for text content and also audio and the video images to make them work together for real-world use cases. And the other one is called graph RAG. Basically use the RAG techniques on knowledge graphs because the graph data is already interconnected. So by using RAG techniques on top of that will make it even more powerful. And I think the last one I want to mention is the AI-powered PCs. Again, AI is coming to everywhere.

So especially in line with the local first architectures that we are seeing in other areas, how much computing can I do locally on the device, whether it's my smartphone or a tablet or IoT device. So this AI is going to be powering the PCs going forward. We already are hearing about Apple intelligence and other similar technologies. But Yes, other than that AI, like you all mentioned GitHub Copilot, so AI is going to be everywhere in the software development life cycle as well.

I had one use case where multiple agents are used for code generation, document generation and test case generation in a software development life cycle. It's only going to grow more next year and be even more like a peer programmer. That's what we always talk about. How can AI be an architect, how can AI be a programmer or how can AI be a QA software engineer. So I think we're going to see more developments in those areas. So that's what I'm seeing. I don't know, Thomas, Shane or Renato, you guys are seeing any other trends in AI space?

Shane Hastie: So I'm definitely seeing the different pace of adoption. As I mentioned right at the beginning, the organizations for whom software is now their core business, but they still think of themselves as not software companies, the banks, the financial institutions and so forth. They're struggling with wanting to bring in, wanting the benefits and really having to tackle the ethical challenges, the governance challenges, but overcoming those, recognizing those, the limited selection of tools that are available. So one organization, the AI tool they're allowed to use is Copilot. Nothing wrong with Copilot, there are 79,000 at least other alternates. But even in that sort of big space, there's a dozen that people should be looking at. And I think one of the risks in that is that they end up narrowing the options and not getting the real benefits.

Thomas Betts: I saw this in my company. We did a very slow rollout of a pilot of GitHub Copilot and they wanted those people to say, "Here, how do we use it? But we're not going to just let everyone go through it". And part of it once they said everyone can use it is you have to go through this training on here's what's actually happening. So everyone understood what are you getting out of it. Things like the hallucinations, you can't assume it's going to be correct, it's only as good as what you give. But if it doesn't know the answer, its job isn't to know the right answer. Its job is to predict the next word, predict the next code. So it's always going to do that even if that's not the right thing. So you still have maybe even more oversight than if you had just doing a full request review or review for someone else's code, right?

We've now done Microsoft Copilot as the standard that the company gets to use. I think this is probably the one you're referring to, everyone can use the generative AI tool to start doing all the things. And because we're mostly on the Microsoft stack, there's the integrations with SharePoint and OneDrive and all of that benefit. So there's reasons to stay within the ecosystem, but again, every employee has to go through just like our mandatory ethics training and compliance training and if you deal with financial data, you have to go through this extra training. If you're going to use the AI tools, here's what you need to know about them, here's how you're safe about that. And I think that training is going to have to evolve a lot year to year to year because the AI that we have in 2024 is not the same AI we're going to have in 2026. It's going to be vastly different.

What are the key trends in cloud technologies? [36:08]

Daniel Bryant: I think it's a perfect segue from everyone talking about AI. Renato, our cloud space has lost its shine. We used to be doing all the innovative stuff, all the amazing things, and now with the substrate powering all this amazing innovation going on in the AI space. So you mentioned already about the sharp skew at re:Invent as one example. There's many other conferences, but the sharp skew at re:Invent towards this AI focus. But I'm kind of curious what's going on outside of that, Renato? What are you seeing that's interesting in general from re:Invent and from CloudFlare and from Azure and GCP?

Renato Losio: I think that the key point that has been going on for already two, three years is that we have moved from really innovation as in any area to really a more evolutionary approach where we had new feature, but there's nothing really super new and that's probably good. I mean we are getting maybe more boring but more enterprise-wise maybe, but that's the direction. Just an example, I mean, you mentioned re:Invent. People tend to go out of re:Invent, say, "This has been the best re:Invent ever". Usually because they get more gadgets and whatever else, but that's usually the main goal talking about sustainability. But even Amazon itself during the keynotes and during the week before re:Invent was highlighting the 10th anniversary of AWS Lambda and 10 years of Amazon Aurora, 10 years of... what was the third one?

I think container service, and then even I think KMS. And those were all announced 10 years ago at re:Invent. If you take a re:Invent today was a great re:Invent, but you don't have something as revolutionary as Lambda. You have cool things, you have a new distributed database, yes, definitely it's cool, but you don't have the same kind of new breaking the world things. It's more evolutionary thing as well in the AI space. That was of course a key part of it. But yes, there were some new financial models for Bedrock the school, they got better names that even someone like myself that is not the skilled in the area, I can say I can get when I should use a light approved or a micro model.

At least I know that the price probably follows that. But apart from that, it's quite, I would say evolutionary. Probably the only exception in this space is Cloudflare, at least the way I see it because probably we used to consider just a CDN. We used to consider networking mostly, but actually in the last few years they're fully fledged cloud provider. Quite interesting services that have been out there at the moment. The other trend I wouldn't say is for 2025, but it is already here, at least in the data space, in the database space, in the cloud database space. I think this was the year that Postgres became the de facto standard. Any implementation of any cloud provider has to be somehow even pretending to be Postgres.

Daniel Bryant: Indeed.

Renato Losio: That's the direction. Even Amazon doesn't mention MySQL for new services for GSQL or even Limitless database earlier this year, used to be their first open source compatible database reference point. Now it's not anymore. All the vector databases are pointing to us as well. So that's the direction I see at the moment.

Daniel Bryant: Fantastic.

Srini Penchikala: Quickly, Daniel, I have a comment. Renato, you are right in saying that Postgres is getting a lot of attention. Postgres has a vector database extension called PG Vector, and that is being used a lot to store the vector embeddings for AI programming. And also the database are becoming more distributed in terms of the architecture and also hosting. So I've been seeing a lot of databases that need to run on on-prem and in the cloud with all the transactional support and the consistency support, so distributed events are kind of helping in this. So definitely like you said, just like cloud is a substrate for everything else to happen, database engineering, data engineering and databases are the foundation for all of these powerful AI programs to work. So we don't want to lose focus on the data side of the things.

What are the key trends in DevOps and platform engineering? [40:34]

Daniel Bryant: I'll cover the platform engineering aspects now. So for me, 2024 has definitely been the year of the portal. Backstage has been kicking around for a while. We had folks like Roadie talking about that. It's got its own colo day at KubeCon. Now the BackstageCon, I've also seen the likes of Port and Cortex emerging, lots of funding going into this space and lots of innovation too. Folks are loving a UI, loving a portal, a service catalog, a way to understand all the services they've got in their enterprise and how these things knit together. Now I've argued in a few of my talks, there's definitely a missing middle, which we're sort of labeling as platform orchestration popping up. And this is the missing middle between something like a UI or a CLI, portal, that kind of good stuff. And the infrastructure layer, things like Terraform, Crossplane, Pulumi, cloud infrastructure in general.

Now I was super excited to see Kief Morris with his latest edition of his book, Infrastructure's Code talking about this missing middle two and also Camille Fournier and Ian Nowland in their platform engineering book that's just been published by Rily. Fantastic read. I thoroughly recommend folks get hold of that, but they were also talking about this missing middle as well. So I'm super excited over the next year to see how that develops. Just in general, I'm seeing platform engineering move more into the mainstream. We're seeing more events pop up. I mean the good folks at Humanitec spun up PlatformCon in 2022. That one is going from strength to strength. There's also Plat Eng Day at KubeCon now and KubeCon in London coming up next year, we're going to see an even bigger Plat Eng day I think with two tracks. So I'm super excited about that. I'm definitely at QCon London.

We're going to dive into platform engineering again, I've got a hat tip my good people that were on the track this year, Jessica, Jemma, Aviran, Ana and Andy did an amazing job talking about our platform engineering from their worlds. Topics like APIs came up, abstractions, automation, I often say three A's of platform engineering, really important. And in particular, Aviran Mordo talks about platform as a runtime. And at Wix they built this pretty much serverless platform and it was a real eye-opener seeing at the scale they're working at how they've really thought about the platform APIs, they've really thought about the abstractions to expose the developers. And a whole bunch of the stuff behind the scenes is automated Now it's all tradeoffs, right? But Aviran, and I saw Andy said the same thing. They're optimizing for developer experience and not just keeping people happy, but keeping people productive too.

And there's lots of great research going on around developer experience. I've got to hat tip the DX folks, Abi Noda and crew and some great podcasts kicking off in that space. And I'm just really interested about that balance, that sort of business balance I guess with proving out value for the platform, but also making sure developers are enjoying their day-to-day work as well. There's a whole bunch of platform engineering FOMO that I see in my day-to-day job and people spinning up platform engineering efforts, spinning up platforms without really having business goals associated with them, which I think is a danger. And I'll hint at some more why I think that's going later on.

What are our predicted trends for software delivery in 2025? [43:25]

Now it's that time where we hold you to predictions you are going to make, and next year the InfoQ bonus is based on success or failure of these predictions. So I'd love to go around the room and just hear what you most excited about for 2025 and predictions and we'll go in the same order if that's all right. Thomas, we'll start with you.

Thomas Betts: Yes, I don't think there's going to be some dramatic shift. There's never dramatic shifts in architecture, but I think the sustainability, the green engineering, I think those concepts are just going to start becoming more mainstream. You look at how team topologies and microservices and all these things overlap. All these books start referencing each other, the presentations start talking about each other in the same ideas in different ways. I think we're going to see architects that just look at it from, "I did this for all of these benefits and I learned to put all these benefits together because they were the right sustainable thing to do and it made my system better". I want to see that presentation at QCon San Francisco next year of we chose to do some architecture with sustainability in mind, and here's the benefits we saw from it. Shane.

Shane Hastie: I'm going to build on the people stuff. I think we're going to see challenges with the return to office mandates. I hope we're going to see some sensibility coming in that when we bring people together, we bring them together for the right reason and that we get the benefit of those human beings in the same physical space. Doing collaborative work generates innovation. You want to allow that, but you also want to give the space for the work that is more effective when we are remote.

So that combination of the two, and there's no one size fits all, and organization shifting away from mandates to conversations and let's treat our people as responsible, sensible adults and trust them to figure out what is going to be the best way of getting stuff done. I want to see the continuing evolution of the team areas, generative AI and other AI tools as partners. I think the agentic AI as a partner is going to be a huge potential and I think we're going to start to see some good stuff happening in that space with the people. But again, the critical thinking, the human skills becoming more and more important. So what's the prediction there? It's maybe more of a hope.

Daniel Bryant: No, I like it, Shane, very positive. It's very good. Srini, on to you.

Srini Penchikala: Yes. Thanks, Daniel. Yes, definitely on the AI side, I can take a shot at a couple of predictions. I think the LLMs are going to become more business domain specific in a sense, just like we used to see our banking standards, our insurance industry standards, I think we'll probably eventually start seeing some finance, FinTech LLM or manufacturing LLM, because that's where the real value is, right? Because a typical ChatGPT program only knows what's out in the internet. It doesn't know exactly what my business does, and I don't want to share my business information out to the outside world.

But I think there will be some of these consortiums that will come together and share at least non-sensitive proprietary information among the organizations, whether it's manufacturing or healthcare. And they will start to create some base language models that the applications in those companies can use to train, right? So it could be the large language model, like Llama 2 will have something more specific on top of that, and then applications will be built on top of that. So that's one prediction for me. And the other one is agents. I think agents, agents and agents. Just like that movie Matrix, agents are coming, right? Hopefully these agents are not as nefarious.

Daniel Bryant: Indeed.

Srini Penchikala: They're not villains, right? Exactly. But Yes, I think we'll see more. I think it's the time for all these generative AI, great content to put into action, not by humans but hopefully by computer programs. So that's another area that I'm definitely looking forward to seeing. Other than that, I think AI will become, like you said, something like a boring cross-cutting concern that will just enable everything. We don't even have to think about it. And maybe just like the toys we buy sometimes, they say batteries not included. Maybe in the future the application that are not using AI, which will be very few, they will say, this application does not include AI, right? Because everything else will be AI pretty much, right? So those are my couple of predictions.

Daniel Bryant: I like it, Srini, fantastic stuff. Renato, over to you.

Renato Losio: Well, first of all, I will say that these are my tech provision in the cloud for 2025 and beyond, as many good ones say. So I will always have the chance next year to say, "Well, there's still time". But I really think that next year will be the first year in the cloud space that for processor Intel won't be the default anymore. Basically we won't consider anymore using Graviton or anything else as the alternative would be the de facto on most deployment.

And the second one, giving as well how different cloud provider implemented now distributed database using their own proprietary network and basically taking advantage of the speed they have. I see the cloud provider going towards this distributed system with less regional focus. As an end user, I will probably... As a developer, I won't carry more long term about the specific region, the geographical area, probably yes, for compliance and many other reasons. But if behind my database is Ohio or Northern Virginia or whatever else, I would probably not care so much.

Daniel Bryant: Thanks, Renato. I like the hedge there or is it a smart move. Well done. So all these, some predictions around the platform engineering space and the good folks at Gartner are saying we're about to head into the trough of disillusionment in their model of adoption. And I also think this is true. My prediction next year is we're going to hear more failure stories around platforms and ultimately that'll be triggered by a lack of business goals associated with the engineering going on. Now, I think this is just part of the way it goes, right? We saw it with microservices, we saw it with DevOps as well.

Ultimately, I think it leads us to a better place. You go into the trough for disillusionment, you hopefully come out the other side on the plateau of productivity and you're delivering business value and it's all good stuff. And I think we're going to bake in a lot of learnings that we sort of temporarily forgotten. This is again, the way of the world.

We often embrace a new technology, embrace a new practice, and we sort of temporarily forget the things we've learned before. And I'm seeing in platform engineering, definitely a lack of focus on business goals, but also a lack of focus on good architecture practices, things like coupling and cohesion. And in particular creating appropriate abstractions for developers to get their work done and also composability of the platform.

So I think in 2025, we're going to see a lot more around golden paths and golden bricks and making it easy to do the right thing to code ship run for developers, deliver business value, and also allow them to compose the appropriate workflow for them. And again, that'll be dependent on the organization they're working on. But I'm super excited to see where platform engineering is going in 2025. As always, it's your pleasure. We could talk all day, all night, I'm sure here, but it's fantastic just to get an hour of everyone's time just to review all these things as we close up the year. I'll say thank you so much for everyone and we'll talk again soon.

Shane Hastie: Thank you, Daniel.

Srini Penchikala: Thank you.

Renato Losio: Thank you, Daniel.

Thomas Betts: Thank you Daniel, and have a happy New Year.

Mentioned:

• Building Green Software (Anne Currie, Sarah Hsu, Sara Bergman)
• Infrastructure as Code (Keif Morris)
• Platform Engineering (Camille Fournier, Ian Nowland)