Cloud Data Systems and Edge AI Make a Major Impact on Today’s Data Science

Gartner outlines the top trends in machine learning and data science, including the impact of generative AI. Learn more about the top trends with our article.

Gartner outlines the top trends in machine learning and data science, including the impact of generative AI.

Investors will pour more than $10 billion into AI startups that use foundation models, tech consultancy Gartner found in its August report on top trends in data science and machine learning. In particular, business leaders are interested in edge and data-centric generative AI, plus using generative AI responsibly.

Jump to:

Cloud-native solutions take over from self-contained software

By 2024, 50% of new system deployments in the cloud will reside entirely within a cloud data ecosystem as opposed to manually integrated point solutions, Gartner predicted. Organizations should look out for converged data and analytics platforms that can solve problems over a wide swath of distributed data. That means businesses will continue to see more cloud-native solutions as opposed to self-contained software or blended deployments.

SEE: Explore the best data science tools for any use case (TechRepublic)

Edge and AI will move analysis toward IoT endpoints

“As machine learning adoption continues to grow rapidly across industries, DSML [data science and machine learning] is evolving from just focusing on predictive models toward a more democratized, dynamic and data-centric discipline,” said Peter Krensky, director analyst at Gartner, at the Gartner Data & Analytics Summit on August 1, as quoted in a press release. “This is now also fueled by the fervor around generative AI. While potential risks are emerging, so too are the many new capabilities and use cases for data scientists and their organizations.”

Businesses are keeping an eye on how AI training and inferencing can help transition data analytics to edge environments near IoT endpoints.

More than 55% of all data analysis by deep neural networks (which generative AI is based on) will occur at the point on the edge where the data is captured by 2025, Gartner said. That’s a large jump from the 10% that occurred in 2021, and speaks to the massive increase in AI adoption since the beginning of 2023.

Cloud and edge computing saw a $84 billion equity investment in 2022, McKinsey found in its 2023 tech trends report.

Data-centric AI can include training on synthetic data

Data-centric AI performs tasks such as AI-specific data management, synthetic data and data labeling to solve problems in accessibility, volume, privacy, security, complexity and scope.

Generative AI can also be used to create synthetic data with which to train other applications. Use cases such as simulating real conditions, predicting future scenarios and removing some risk from AI will boost the amount of synthetic data in industry to 60% by 2024, Gartner predicts. That’s up from just 1% in 2021, likely driven again by the commercialization of generative AI in late 2022 and early 2023.

Responsible AI follows the trend toward regulation

Responsible AI is a philosophy that takes into account business and societal value, risk, trust, transparency and accountability, in regards to generative AI. Gartner pointed out that “The concentration of pretrained AI models among 1% of AI vendors by 2025 will make responsible AI a societal concern” – meaning that a problem in one model could rapidly spread across a massive number of clients. Concerns include generative AI introducing factual mistakes into content — accidentally because of hallucinations or on purpose as part of a planned misinformation campaign. People are also concerned about AI introducing bias into content or plagiarizing copyrighted work.

Some organizations, including Salesforce, support recent conversations about government regulation on generative AI.

“Salesforce supports tailored, risk-based AI regulation that differentiates contexts and uses of the technology and ensures the protection of individuals, builds trust, and encourages innovation,” Salesforce’s executive vice president of government affairs, Eric Loeb wrote in a blog post in July.

AI investment continues to rise

Overall, Gartner found that more than $10 billion will have been invested in AI startups that rely on foundation models by 2026. More organizations will implement AI solutions, and more industries will take AI technologies and AI-based businesses into account.

In a pool of 2,500 executive leaders in May, Gartner found that 45% said the hype around ChatGPT spurred them to put more money into generative AI. Most (70%) of those organizations are still exploring their options, while 19% have a pilot program or have put use of generative AI into production.