A young boy looking through a telescope at Lands End in Cornwall, England, United Kingdom. (Photo by: Education … [+]
Technological viewpoints overlap. This means that even as we experience the new generation of Internet of Things (IoT)-based cameras, sensors, and tracking devices, we ourselves spend time examining this layer of technology as human beings , often simply to assess our level of confidentiality and anonymity.
A similar duality exists in the world of AI. Even as we realize that artificial intelligence (AI) monitors a percentage of our activities, choices and actions in the real world and online for positive good, we must also be sure to examine the functions of AI for determine the origin. , observe their state and status, observe their behavior, and evaluate the validity of the decisions they make.
Embryonic prototyping
This AI observability process is a fluid and embryonic area, for many reasons, not least because the entire AI deployment surface itself is still relatively experimental for many companies. But even at this stage, comprehensive analytics tools and observability services are being developed to meet this critical need. Among the technology providers eager to quickly assert their skills in this area is Dynatrace. The company describes itself as a unified observability and security company with an extensive analytics and automation platform.
Terms like unified and holistic are of course often overused, especially in the information technology field. So, does it have any value when used in this context?
Dynatrace uses the term unified to describe an ability to provide observability across data, code, IT connection points from application programming interfaces (APIs) to containerized software component services to the web and to the cloud in the broad sense… and of course through the applications and data services themselves. Today, in the current era of generative AI and the supporting repositories and infrastructure it requires from large language models (LLM) and vector databases, the company also includes these channels of information in its definition of what constitutes a unified and holistic view.
As a new development for this year, Dynatrace has enhanced its platform with a specific view (observability view pun) through generative AI and large language models.
“Generative AI is the new frontier of digital transformation,” said Bernd Greifeneder, CTO at Dynatrace. “This technology enables organizations to create innovative solutions that drive productivity, profitability and competitiveness. While transformational, it also poses new challenges in security, transparency, reliability, experience and cost management. Organizations need AI observability that covers all aspects of their generative AI solutions to overcome these challenges. Dynatrace is expanding its observability and leadership in AI to meet this need, helping customers confidently and securely adopt AI with unparalleled insights into their AI-driven generative applications.
The end-to-end AI stack
Now a branded and produced technology, Dynatrace AI Observability is meant to cover the AI stack end-to-end. So do we have a whole AI stack now? Yes. Enterprise is everything that is connected, responsible for serving, driving and executing the AI components we now want to merge, including infrastructure elements, such as hardware like processing units graphics (GPU) Nvidia, the fundamental models (the basic LLM models that developers use). to start) like GPT4… then towards “semantic caches” (see below) and vector databases, such as Weaviate, as well as orchestration frameworks, such as LangChain.
Interesting fact and like defined succinctly here by vector database company Zilliz, “Semantic Caching Stores [look after] data based on its meaning, meaning that two queries with the same meaning will return the same result, even if the underlying data has changed. This can be useful for complex queries involving multiple tables or data sources.
It also supports leading platforms for building, training, and delivering AI models, including Microsoft Azure OpenAI Service, Amazon SageMaker, and Google AI Platform. Dynatrace AI Observability uses the company’s Davis AI and other core technologies to provide an “accurate and comprehensive view” of AI-driven applications. As a result, organizations can deliver excellent user experiences while automatically identifying performance bottlenecks and their root causes.
The question now is how much confidence software application development engineers can have when coding AI applications and, in equal measure, how much confidence we as users can have when we start integrating these new smart apps into our lives at home and at work?
“Let’s be realistic and pragmatic about the current state of AI. It’s still in its infancy in terms of deployment in many organizations, which means it’s in its “early stages” in terms of the tools being used, the extensive language models that are being used. ” clarified Steve Tack, senior vice president of product management at Dynatrace. “Because that’s where we are, Dynatrace AI Observability was built and designed to provide a way to deploy high-performance, secure AI applications. Any given AI function is usually part of a larger service, so it’s important to remember that there’s a lot to it. momentum and cadence in how AI is created – if things remained static in technology then we as a company probably wouldn’t exist, but that’s not the case …then we exist,” he enthused.
More than just a symbolic gesture from AI
We’ve said from the beginning that the whole AI observability game involves being able to make sure that we’re looking at AI functions to determine where they’re coming from. As such, Dynatrace AI Observability with Davis AI (the company’s AI engine) helps businesses comply with privacy and security regulations and governance standards by accurately tracing the origins of results created by their applications. Additionally, it helps predict and control costs by monitoring the consumption of “tokens,” which are the basic units used by generative AI models to process requests.
A bit nerdy and geeky (in a good way), but worth knowing for the next AI conversation someone might find themselves in when the topic of tokens comes up, tokenization methods ( as reinterpreted from OpenAI and ChatGPT news) can be briefly summarized as follows:
- Space-based tokens: Text is tokenized based on space usage, so “I read Forbes” would be made up of three tokens: I, read, Forbes.
- Dictionary-based tokens: Tokens are created for each word used that matches an existing record in a predefined dictionary. So “I read Forbes” would issue three tokens, one for each commonly understood word, in much the same way as our first example with spaces. .
- Subword Tokens: Easy to understand, “I like reading Forbes” would be made up of six tokens: I, I, I enjoy, I read, Forbes.
- Byte-Pair Encoding (BPE) Tokens: Tokens are defined by the number of bytes and it is a technique that was first developed as an algorithm in order to compress text strings into larger values short – returning the text to its original form once the tokenization is known. as standardization – but that’s a complex story for another day.
Keen to detail a new partnership, Ali Dalloul, vice president of AI at Microsoft, notes that the Azure OpenAI service (a generative AI product) now aligns with Dynatrace AI Observability to provide shared customers with all the information detailed here. “This powerful combination helps ensure these services meet the highest standards of security, reliability and performance while allowing the teams that manage them to control costs,” Dalloul said.
Analyst firm Gartner suggests that by 2028, AI adoption will peak with more than 50% of cloud computing resources devoted to AI workloads, up from less than 10% in 2023. The suggestion The broader one here is that many organizations are concerned about the associated costs. with generative services based on AI; often because they can be several times more expensive than traditional cloud services and are difficult to predict because they rely on the consumption of generative AI tokens by applications that are not yet in production.
As governments around the world now establish regulations focused on the responsible and ethical use of AI technologies (without the risk of AI-related hallucinations and biases and much more) and in compliance with applicable laws , the need to monitor and observe AI components has almost certainly never been great.
IT monitoring culture
This whole story is about a different way of using technology compared to how we did it in pre-millennial times.
While many of us haven’t really had the proximity we have today to “apps in our pocket” given the ubiquity of the smartphone, the time we’ve spent with technology hasn’t not seen acting with as much analytical curiosity towards platforms. and the tools we used.
We plugged it in, set it up and turned it off, for the most part.
Today the acceptance of IT has of course changed, we all understand the existence of internet scams, ransomware and automation to one degree or another and the arrival of generative AI was not done without asking questions. As we, the users, now more closely “monitor” the technology we use, it is perhaps comforting to know that system-level, AI-centric monitoring and observability tools are on the way development to provide a lower viewing lens.