A rise in database complexity and the drive for faster development is leading to significant skills gaps for IT teams in 2024

Increasing complexity, the rapid adoption of emerging technologies and a growing skills gap are the biggest concerns facing IT leaders in 2024, according to The State of the Database Landscape, a major new survey from end-to-end Database DevOps provider Redgate.

The research paints a picture of teams worldwide working at pace, with nearly a third (30%) of development teams now deploying database changes to production in a day or less, up by 50% from just two years ago. Further complicating the scenario, the survey showed the majority (79%) of IT teams are now using more than one database platform, with 29% of respondents using more than five. However, across a wide range of categories, the 3,800 IT professionals surveyed cited a lack of skills, knowledge gaps and a rapid need for the upgrading and diversification of abilities at every level.

"Catering for different use cases, adding flexibility and achieving cost savings are the driving factors behind the escalating pace of change toward a multi-platform database landscape," said David Gummer, Redgate Chief Product Officer. "However, the sheer volume of platform choices, with respondents citing usage of 16 different database types, highlights why it's critical that IT teams are upskilled and have the right tools in place. It's clear that organizations are currently scrambling to keep up with increased complexity, the pressures of compliance and emerging technologies like AI and the cloud, and are seeking solutions to narrow the skills gap."

The gaps start with multiple databases in use

Despite the many benefits of employing multiple databases, an increased skills requirement for individuals and teams is cited as the number one challenge people face, with 38% of respondents naming this as a problem. Skill diversification is also cited as a top need by 31% when dealing with data management processes across multiple database types.

While Database DevOps is well established, with 73% of respondents stating they have already adopted it across all or some of their projects, or are planning to do so in the next two years, issues remain. More than a third of respondents (42%) state there is a lack of critical skills in their teams, while nearly a third (29%) also cite a limited understanding of DevOps as a major obstacle.

"It's good to see the database now included as a natural partner in DevOps, with the alignment of people, technology and processes leading to faster, more efficient and stable application development," Gummer continued. "However, to keep pace with business demands, we have to overhaul our thinking and invest in cross-team education as well as tools across the entire database development lifecycle, particularly if we hope to be ready for yet more new technologies which are already at our doors."

The gaps continue with emerging technologies

The use of Artificial intelligence (AI) in database development is also on the rise with 20% of respondents to the survey currently using AI for database management and a further 35% considering doing so. For respondents putting AI to work in database management, 57% confirmed using AI to analyze and optimize database queries or code, 48% said they used it for automating testing scenarios, and 55% confirmed using AI to generate sample data or code.

While there are advantages to be gained, respondents also raised concerns about AI, including worries about data security (42%), the accuracy of the results AI tools produce (37%), as well as concerns over a lack of training and expertise (28%).

The move to cloud is firmly established, with 88% of organizations now using cloud in some way. Although the survey revealed that over 1 in 3 respondents (36%) favored a hybrid approach, it's clear cloud adoption is poised to gain further momentum with those hosting databases mostly or all in the cloud growing from 18% to 36% over the last three years.

While cost is typically seen as the biggest driver to the cloud, respondents noted that the scalability and flexibility it offers (48%) followed closely by high availability and reliability (45%) were the two biggest reasons for migrating workloads and data.

"2024 will bring a renewed need for DBAs and developers to acquire new skills so they can best understand and control costs, manage security more effectively, and monitor databases for performance issues when estates are spread across one or more cloud providers as well as on-premises," Gummer concludes. "In effect, database professionals need to double their skillset to manage the range of database types that are now used by nearly 80% of all respondents in the survey. As an industry, we must work together and rise to the challenge of narrowing these skills gaps ? whether that means upskilling internally or finding external solutions to address the needs across the Database DevOps landscape."

Learn More about the State of the Database Landscape:

Survey page: The State of the Database Landscape Survey
Webinar: Addressing the Technology Skills Gap

About The State of the Database Landscape Survey

Conducted in late 2023, the State of the Database Landscape Survey attracted over 3,800 responses from Developers, DBAs, IT professionals and the C-level, covering every sector and company size across the world. This is the sixth survey Redgate has run on the topic.

About Redgate Software

Redgate offers end-to-end database DevOps to help organizations streamline software development and get value from their data faster. Redgate's portfolio of solutions helps organizations reliably solve the complex challenges of database management across the DevOps lifecycle on any database, anywhere. We make life easier for IT leaders, development and operations teams, increasing efficiency, reducing errors and safeguarding business-critical data. More than 200,000 people globally, including 92% of the Fortune 100, trust Redgate to deliver ingeniously simple software.