1、DATA TRENDS 2024DATA TRENDS 20247 Ways Leading Organizations Are Building Toward Advanced AI Success2Executive Summary.3Introduction:Advancing in the AI Age.5Cementing the Data Foundation.7Trend One:Python Is the Language of Choice for AI Programming.8Trend Two:Enterprises Are Tapping Their Unstruct

2、ured Data.9Trend Three:Enterprises Are Getting More Granular in Their Data Governance.10AI Scales With Apps.12Trend One:The Democratization of AI Is Here.13 Trend Two:The LLM Explosion Is Happening NowProbably at Your Office .14 Trend Three:The Chatbot Is on the Rise.15 Trend Four:Enterprises Want A

3、pps and Data Within a Unified Data Platform.16 From Foundation to Elevation.17 Next Steps.18 Appendix:Methodology.19 TABLE OF CONTENTS2DATA TRENDS 20243DATA TRENDS 20243EXECUTIVE SUMMARYWe looked at how more than 9,000 Snowflake accounts adopted features and capabilities of the Data Cloud over the p

4、revious fiscal year to reveal trends,both in terms of the foundational development of data infrastructure and those users first moves into advanced AI.Generally,we compared January 2023 to January 2024 to align with Snowflakes fiscal year,except in cases where features went into public preview durin

5、g the year.In those cases,we compared the first full month in public preview to January 2024.For the full methodology,see the appendix.Highlights from this report include:FIRMING UP THE DATA FOUNDATION1 Python is the language of choice for AI programming.With its ease of use,active community and eco

6、system of libraries and frameworks,Python use grew 571%,considerably more than any other language year over year.Python skills will be increasingly essential to development teams as they venture into advanced AI.2 Enterprises are finally tapping their unstructured data.Most dataas much as 90%by some

7、 estimatesis unstructured videos,documents and more.We saw processing of unstructured data grow by 123%.Thats good news for many uses,not the least of which is advanced AI.Proprietary data will give large language models their edge,so unlocking that underutilized 90%has huge value.3 Enterprises are

8、getting fine-grained about their data governance.Were seeing not just more governance measures applied to data;were seeing a more refined approach as organizations embrace a wide range of tagging standards and features.The takeaway:While usage of every data governance feature rose 70%-100%,the numbe

9、r of queries against protected objects is up 142%.Governance is not about locking down your data,its about making it more available for secure,authorized uses,and were seeing exactly that.4DATA TRENDS 2024MAKING AI ACCESSIBLE1 The democratization of AI is here.A major promise of AI is that it will m

10、ake technology available to less technical users.Weve empowered that through the machine learning functions of Snowflake Cortex,and since public preview of key features began in June 2023,weve seen the number of active accounts adopting ML-based functions grow by 67%.That opens up more possibilities

11、 because data scientists and other experts are no longer a bottleneck.2 The LLM explosion is happening nowprobably at your office.What bottleneck?In the last fiscal year in the Streamlit developer community,we saw 20,076 unique developers work on 33,143 LLM-powered apps.That means that the future fi

12、lled with the power of AI is here.It may not be evenly distributed yet,but its here.3 The chatbot is on the rise.Single-text input LLM apps may be easier to make,but they dont allow refinement through natural conversation.For that you need chatbots,and increasingly thats what the devs are making.Fro

13、m May 2023 through January 2024 in the Streamlit community,chatbots went from 18%of LLM apps to 46%.And climbing.4 Enterprises want apps and data within a unified data platform.We make it possible for users to build applications within our data platform,where their data resides,via the Snowflake Nat

14、ive App Framework.Maybe its the ease of use or the single source of truth.Maybe its the security and governance advantages.But the data shows that people want to bring the work to the data.The number of Snowflake Native Apps grew 311%,and the use of those apps is up 96%,based on January 2024 utiliza

15、tion compared to July 2023(Snowflake Native Apps went into public preview on June 27,2023).5DATA TRENDS 2024Were now a year and a half into the generative AI era,and things are only accelerating.OpenAIs release of ChatGPT and then GPT-4,Metas decision to open source Llama and Llama 2,and a host of o

16、ther announcements and innovations around the application of advanced AI have stirred more excitement and driven real progress in the development and enterprise adoption of large language models.Tremendous opportunities and challenges lie ahead,and as we analyzed use of the Snowflake Data Cloud to u

17、nderstand the latest trends around data and technology,our chief interest was around how enterprises are preparing for an unfolding era in which advanced AI accelerates and transforms how they do business.The Snowflake Data Cloud encompasses data,models and applications from thousands of organizatio

18、ns across many industries.Looking at how they work within the platform,including which features they use,paints a vivid picture of the decisions being made to deal with current challenges and prepare for future success.ADVANCING IN THE AI AGE6DATA TRENDS 2024A lot of industry research surveys execut

19、ives and practitioners,asking them to estimate things such as what percentage of their data is unstructured,or to describe how confident they feel about their approach to data governance.This report didnt ask anyones opinion.Instead,we looked at how enterprises worldwide are making decisions and app

20、lying their resources to leverage their data.Through that lens,a picture emerges about how the modern,data-forward enterprise is shaping its data strategy on the cusp of an AI revolution.In short,business and technology leaders at these organizations are preparing for the future.They are taking init

21、ial steps into the world of large language models and generative AI.More importantly,they are fortifying their data foundation.While the specific technologies around advanced AIthe algorithms and appsare powerful,they dont work alone.To be successful,a business must build the shiny,new AI technology

22、 on top of a solid stack of organizational practices and technologies to ensure a companys data is available,secure and properly governed.In other words,the LLM is the dessert,while a solid data infrastructure is the main course.In our predictions report for 2024,our in-house experts advised that th

23、e proper response to the new AI age is not to desperately create a new data strategy,but to accelerate the same solid,thoughtful practices you were following before you ever heard of ChatGPT.When we look at how Snowflake users are working with their data,we see exactly that:a focus on silo-busting,r

24、efining governance practices,and finally coming to grips with the flood of unstructured data.For starters.The generative AI era does not call for a fundamental shift in data strategy.It calls for an acceleration of the trend toward breaking down silos and opening access to data sources wherever they

25、 might be in the organization.”JENNIFER BELISSENT Principal Data Strategist,in Snowflake Data+AI Predictions 2024DATA TRENDS 20247Organizations are doing a lot to make more data securely,appropriately available to todays tools and applications as well as tomorrows(or next weeks)AI advance.At the fou

26、ndation layer,weve identified the following three trends as significant in the past year.On their own,each of these trends is a singular data point about how IT organizations are handling various challenges.Taken together,they suggest a larger story about how CIOs,CTOs and CDOs are modernizing their

27、 organizations,embracing AI experimentation,solving data problems and driving resource-stretching efficienciesall necessary steps to meet the opportunities of advanced AI head-on.CEMENTING THE DATA FOUNDATIONTREND ONE:PYTHON IS THE LANGUAGE OF CHOICE FOR AI PROGRAMMING Developers are able to work wi

28、th a variety of programming languages in Snowflake,and its with interest that we note which languages are growing in popularity.In the past year,Python has surged.Python has a lot going for it,including:Its easy to learn and read,letting developers focus on solving AI problems rather than parsing ab

29、stract syntax.It has a vast ecosystem of libraries and frameworks that simplify potentially daunting AI tasks,from implementation of neural networks to natural language processing.It has a big,active community of contributors,which accelerates learning and problem-solving.Its flexible and portable,s

30、o developers can deploy AI applications across different platforms,systems and environments.Its extensive data-handling capabilities make it easy to manipulate data,which is a core challenge of any AI/ML project.Overall,Python lets devs focus on the problem,not the language.They can work fast,accele

31、rating prototyping and experimentationand therefore overall learning as dev teams make early forays into cutting-edge AI projects.And in the Snowflake Data Cloud,devs are seriously embracing Python.In Snowpark,which expands programmability in Snowflake,Python use grew considerably faster than both J

32、ava and Scala in the last fiscal year:Python grew by 571%,while Scala grew by 387%and Java grew 131%.+571%+387%+131%PYTHONSCALAJAVADATA TRENDS 20248AI/ML IS GROWINGWITH PYTHONAs Python use skyrockets in Snowpark,usage of some of the most popular AI/ML open source Python libraries in Snowpark has inc

33、reased by 335%,including:Developers are bringing more AI/ML work to Snowflake,because they need a unified data platform and access to huge amounts of data used to build,train and run advanced models.But we believe the increase represents not only a shift of existing work to our platform,but a net in

34、crease in experimentation with advanced AI.AI-friendly Python significantly outpaced Scala and Java growth in the Data Cloud.474%SCIKIT-LEARN IS UP357%XGBOOST IS UPMost data is unstructured,and most enterprises struggle to do much with it.This is not a problem thats going to go away.According to IDC

35、,90%of the data generated by organizations in 2022 was unstructured.1Extracting value from that data has been a tech challenge for years,exacerbated by the near-simultaneous arrivals of smartphones and social media,and complicated by evolving regulatory regimes and privacy practices that govern all

36、of an enterprises data,structured or not.That last point is important;even as automation and artificial intelligence help us extract meaning from unstructured data,the actual management of it becomes more difficult.Despite the challenges,Snowflake users are getting value out of unstructured data,esp

37、ecially with the growth of AI/ML.These data types are being processed with Python,Java and Scala,languages commonly used by data engineers,data scientists and app developers.The suite of languages for unstructured data processing became publicly available in public preview or general availability on

38、 June 27,2023.Given that Python in particular is the language of choice for many developers,data engineers and data scientists,its fast-growing adoption suggests that these unstructured data workflows are not just for building data pipelines,but also involve AI applications and ML models.TREND TWO:E

39、NTERPRISES ARE FINALLY TAPPING THEIR UNSTRUCTURED DATA9DATA TRENDS 2024+675%1.IDC White Paper,sponsored by Box,“Untapped Value:What Every Executive Needs to Know About Unstructured Data,”IDC#US51128223,Aug 2023+123%UNSTRUCTURED DATA PROCESSING OFFROM JULY 2023 TO JAN.2024The last foundational trend

40、is certainly not the least.Governance is absolutely essential to data strategy broadly,and AI strategy in particular.The outputs of LLMs and generative AI can be inaccurate or inappropriate,and a strong governance regime helps limit negative surprises.In last years trends report,we noted that with b

41、oth data regulations and consumer privacy concerns on the rise,we had seen increased adoption of data governance features.In short,we saw that our users were applying more tags governing access and use of their data,meaning that they were ensuring that necessary audiences could make use of their dat

42、a while restricting unauthorized user access.This year,that trend continues and in fact deepens.Weve seen significant increased adoption of governance features in a way that indicates not merely restriction,but control.The wide embrace of multiple governance features suggests that users want granula

43、r control over data to make it appropriately available to more users,for more use cases.This refined control is necessary to responsibly unlock the value of sensitive data.TREND THREE:ENTERPRISES ARE MORE GRANULAR IN THEIR DATA GOVERNANCEDATA TRENDS 202410Among the indicators of a more granular appr

44、oach to data use,we saw use of the following governance features rise year over year:The number of tags applied to an object rose 72%.The number of objects with a directly assigned tag is up almost 80%.The number of applied masking or row-access policies increased 98%.The number of columns with an a

45、ssigned masking policy grew 97%.The cumulative number of queries run against policy-protected objects is up 142%.That last stat is particularly significant.Theres a popular misconception that governance is about saying no,that it slows down or limits data innovation.While good governance is meant to

46、 put the brakes on genuinely unsafe or inappropriate activities,its also an enabler of effective,responsible data usage.Were seeing more and more governance through the use of tags and masking policies,but the amount of work being done with this more carefully governed data is rising rapidly.We expe

47、ct these trends to continue as more and more enterprises improve how they govern their data,increase their responsible usage of it,and reap the benefits that data provides to their bottom line.CUMULATIVE NUMBER OF JOBS RUN AGAINST POLICY-PROTECTED OBJECTS,11DATA TRENDS 2024+142%AI SCALES WITH APPSWh

48、ile the establishment of a solid data platform and a strategy that breaks down silos and finds efficiencies has been a well-understood goal for years,AI is still mostly untapped by the enterprise.In the year that LLMs and generative AI have been in the media glare,many enterprises have begun to expe

49、riment,launching initial projects.Within the Snowflake Data Cloud and the Streamlit community,were able to measure activity in the LLM space and around application development,and throughout 2023 we saw great enthusiasm to get to work.As with the foundational section,weve identified four trends in t

50、hese early days of advanced AI.A challenge of measuring trends in the enterprise AI space is that theres no precedent.In some cases,we made features available during 2023,so we dont have years of previous data to compare.What we have seen is enthusiastic uptake,and patterns of preference that we thi

51、nk point the way for these early days.12DATA TRENDS 2024TREND ONE:THE DEMOCRATIZATION OF AI IS HEREA significant promise of LLMs and generative AI is that you dont have to be a highly trained data scientist to work with them.Natural language interfaces mean that you can talk to the dataor rather,the

52、 app that sits on top of the datalike a human,and the data/app will deliver its answers in a reasonable approximation of human conversation,too.That amounts to a“democratization of AI,”as the tech marketers like to say.And its here.While this years report does not have year-over-year statistics,what

53、 we saw in 2023 was tremendous,widespread enthusiasm.The fast adoption of the ML-based functions available in Snowflake Cortex shows how fast AI can happen when there is a solid foundation of data in place.These functions make it easier for those who arent data scientists to work with machine learni

54、ng algorithms.The number of active accounts using ML-based functions2 grew 67%between July 2023(the first full month after public preview)and January 2024.That surge of initial growth,sustained over the remaining six months of the fiscal year,indicates the enthusiasm for,and the utility of,these“dem

55、ocratizing”functions.Comparing July 2023 to January 2024,monthly usage grew 90%.These are early days,and of course that growth surge starts from a relatively small initial point,but were excited to see sustained and growing interest in tools that put more and more of the power of advanced AI into th

56、e hands of less-technical users.This frees the relatively small(and overwhelmed)teams of data scientists from being a bottleneck,and allows those experts to concentrate on the most complex and high-value projects.JULY 2023JAN 2024USAGEACCOUNTS13DATA TRENDS 2024+90%+67%2.ML-based functions evaluated

57、for this report include anomaly detection,forecasting and contribution explorer,which all went into public preview on June 27,2023.Anomaly detection and forecasting were subsequently announced into general availability on Dec.18,2023.Since ML-based functions became available in late June,more adopti

58、on by user accounts,and rising overall usage,indicate early steps toward the democratization of AI.Note:Growth was not linear.This graphic illustrates the difference between the start and end points.TREND TWO:THE LLM EXPLOSION IS HAPPENING NOWPROBABLY AT YOUR OFFICE20,076DEVS WORKED ON33,143LLM-POWE

59、RED APPS IN9MONTHSWhen generative AI and LLMs became the singular topic of tech conversations a year and a half ago,we were assured that this technology would be everywhere,infiltrating every aspect of how we live and work.We cant say that this reality has fully materialized yet,but were definitely

60、seeing a lot of effort to get us there ASAP.Within the Streamlit developer community,between April 27,2023,and Jan.31,2024,we saw 20,076 unique developers work on 33,143 LLM-powered apps(this includes apps that are still in development).Historically,the Streamlit community has had a large percentage

61、 of non-corporate users,so we wondered if this massive surge might mostly be solo experimentation.But in a survey of 1,479 respondents,nearly 65%said their LLM projects were for work.And it seems that these developers are steadily improving their creations.Vector databases and vector search help imp

62、rove the creativity and utility of an LLM app by making connections between related concepts rather than requiring exact word matches.The result is smarter,more accurate outputs,faster.14DATA TRENDS 2024TREND THREE:THE CHATBOT IS ON THE RISEThe great thing about a conversational interface is that yo

63、u can have a conversation.Weve seen in recent months a decided shift from the easy-to-build,straightforward single-text-input LLM toward the chatbot,which allows refinement through iterative text input.Looking again at the more than 20,000 LLM-powered apps being developed with Streamlit,we see a def

64、inite direction for the chatbot,and its up.In the week starting April 30,2023,single-text-input apps peaked at 82%of all LLM apps built with Streamlit,leaving 18%for the chatbots.A SNAPSHOT OF DEV CONCERNSIn a community-wide survey,more than 980 Streamlit users selected their top concern,from a list

65、 of four common worries,about working with LLMs.The results were:TRUST:Is the LLM response accurate?36%PRIVACY:Is my data safe?28%COST:AI aint cheap!19%SKILLS:Im still learning 17%From that point,the single-input line trended down and the chatbot line rose.By the end of January 2024,chatbots account

66、ed for 46%of LLM apps,with single-input apps comprising 54%.The steady climb of the chatbot probably does not represent a shift in the markets appetite for LLM apps.More likely,developers are increasingly able to make more complex chatbot apps to offer greater versatility and interactivity to meet b

67、oth business needs and user expectations.MAY0%20%40%60%80%JUNJULAUGSEPTOCTNOVDECJAN2024WEEKLY%OF THE TOTAL USAGEARE CHATBOTS THE FUTURE?SINGLE TEXT INPUTCHATBOTDATA TRENDS 202415TREND FOUR:ENTERPRISES WANT APPS AND DATA WITHIN A UNIFIED DATA PLATFORM FOR BETTER SECURITY AND GOVERNANCEYou dont have t

68、o build your LLM application on the same platform as your data,but there are significant advantages to doing so.By having unified data governance and not having to move data across compute environments,application development is faster,deployment is easier,and operational maintenance costs are lower

69、.Therefore,to continue practicing what we preach about bringing the work to the data,rather than vice versa,we introduced the Snowflake Native App Framework in 2023.Snowflake Native Apps let users deploy applications within the Data Cloud,leveraging the Snowflake platform to run all three layers of

70、the app,including data,processing and user interface.But the question is,does anyone actually want that?CYBERSECURITY WORK IS MIGRATING TO THE DATA PLATFORMUnderscoring the trend to bring work to the data,were seeing a rise of cybersecurity workloads being brought to the Snowflake Data Cloud.For cyb

71、ersecurity connected apps,where a SaaS vendor stores and processes data in the end consumers Snowflake instance,the average number of connected accounts increased 72%year over year.This tells us that cybersecurity teams see the value of doing security work within their companys unified data platform

72、,rather than through externally managed applications.The early answer appears to be“Yes.”The Native App Framework went into public preview on June 27,2023.Comparing July 2023 to January 2024:Weve seen 311%growth in the number of Snowflake Native Apps published.We saw 147%growth in installation/adopt

73、ion of these applications.Usage of these apps grew 96%.What this means is that,given the choice,users want to build applications within their data platformwhere the data israther than export copies of the data to external technologies.And frankly,it makes sense.Weve seen that a strong data foundatio

74、n prepares an organization to succeed with AI.That enterprises would want to work within a solid data platform to create their applications is an extension of that principle.We believe this will soon be an industry-wide baseline.DATA TRENDS 202416FROM FOUNDATION TO ELEVATIONIT teams are used to how

75、much work occurs on the backend to provide a positive,painless experience.The simplest application hides a lot of complexity.Thats definitely true with LLMs and generative AI.Were seeing that organizations understand this and are fortifying their data foundation even as they make their first forays

76、into cutting-edge AI.Some of the foundational trends were seeing apply directly to AI:robust,refined governance;increased use of Python;coming to grips with the vast quantities of unstructured data.Others speak to a general excellence and willingness to adopt new practices to accelerate time to valu

77、e,such as the growth of serverless computing.As organizations progressively improve their foundation,they pave the way for successful AI initiatives that will deliver reliable,ethical,secure and impactful results.And the trends were seeing in the AI and applications spaces suggest progress is being

78、made.Organizations are picking their models,creating more complex LLM applications,making AI more available to a wider range of users,and reaping the benefits of a unified data platform.There has been a lot of hype around the transformational potential of AI,but judging from what were seeing in the

79、Data Cloud,the frenzied fanfare is beginning to materialize into concrete results.DATA TRENDS 202417Learn more about how Snowflake can help you improve your data foundation and launch successful AI initiatives.SNOWFLAKE FOR AI AND MLSee how you can securely build and deploy LLMs and ML models in the

80、 Data Cloud.SNOWPARKRuntimes and libraries that securely deploy and process Python and other programming languages in Snowflake.NEXT STEPSSNOWFLAKE HORIZONSnowflakes built-in governance solution provides a unified set of compliance,security,privacy,interoperability and access capabilities in the Dat

81、a Cloud.STREAMLIT IN SNOWFLAKETurn data and ML models into interactive apps with Pythonnow all in Snowflake.LEARN MORELEARN MORELEARN MORELEARN MORE18DATA TRENDS 202419The Snowflake Data Trends Report 2024 is generated from fully aggregated,anonymized data detailing usage of the Snowflake Data Cloud

82、 and its integrated features and tools.In this report,we examine patterns and trends in data and AI adoption across more than 9,000 global Snowflake accounts.The Snowflake Data Cloud provides insight into the state of data and AI,including which technologies are the fastest growing.Note that usage a

83、ttributable to internal consumption,if any,has been removed and is not reflected in any of the metrics contained herein.The accounts and usage reflected in this report represent every major industry and include both longtime Snowflake users and others who only recently joined the Data Cloud.Except w

84、here noted in the text,the data in this report compares monthly averages from January 2024(represented as“this year”)to averages in January 2023(“last year”).When compared,this is depicted as“year over year”growth to align with Snowflakes fiscal year end,though the figures themselves are only repres

85、entative of January figures to calculate growth.When possible,we have provided these year-over-year comparisons to showcase growth trends over time.Where data was drawn from Snowflake features that became publicly available after the start of the fiscal year,data was collected and compared as of the

86、 first full month after which the feature became available in public preview,and that date is noted in the text.Notably,growth figures for features moving into public preview are expected to be considerably higher,as private previews are limited in scope and necessarily restricted to select Snowflak

87、e customers.APPENDIX:METHODOLOGYABOUT SNOWFLAKESnowflake enables every organization to mobilize their data with Snowflakes Data Cloud.Customers use the Data Cloud to unite siloed data,discover and securely share data,and execute diverse artificial intelligence(AI)/machine learning(ML)and analytic wo

88、rkloads.Wherever data or users live,Snowflake delivers a single data experience that spans multiple clouds and geographies.Thousands of customers across many industries,including 691 of the 2023 Forbes Global 2000(G2K)as of January 31,2024,use the Snowflake Data Cloud to power their businesses.Learn

89、 more at 2024 Snowflake Inc.All rights reserved.Snowflake,the Snowflake logo,and all other Snowflake product,feature and service names mentioned herein are registered trademarks or trademarks of Snowflake Inc.in the United States and other countries.All other brand names or logos mentioned or used herein are for identification purposes only and may be the trademarks of their respective holder(s).Snowflake may not be associated with,or be sponsored or endorsed by,any such holder(s).