Skip to content

/articlesIT operations

Ask Dynatrace and Splunk in one conversation

Connect Botify to Dynatrace and Splunk and ask one question: it lists problems and deployments, searches logs and traces, and only reads unless you approve.

Updated
Published
min read
06
On this page
08
On this page08

Why connect one assistant to both Dynatrace and Splunk?

Many IT teams run both. Dynatrace watches applications and infrastructure and opens problems on its own. Splunk holds logs, security events and long history. During an incident, engineers switch between the two, write the same question in DQL (Dynatrace’s query language) and again in SPL (Splunk’s), and match hosts and services by eye.

Botify ends that switching. It keeps one question and one time window, asks each platform through its API and puts the results in one answer. It does not copy your monitoring data anywhere: it reads the platforms you already pay for, with the permissions you give it.

What can you ask Dynatrace and Splunk together?

The best questions are the ones your engineers already ask in the incident channel, where the answer sits in both tools.

  • Which problems are open on the payments service right now, and what does Dynatrace say caused them?
  • Was anything deployed to checkout in the two hours before the slowdown?
  • Show error logs from the order-api hosts since 09:30, grouped by error message.
  • Which Splunk alerts fired overnight, and does any match an open Dynatrace problem?
  • Are there ITSI notable events for the same service, and when did they start?
  • Which checkout requests were slowest in the last hour, and which downstream call took the time?

What does Botify do in Dynatrace?

The Dynatrace integration has 14 actions, each with its own permission and risk level. Thirteen only read and run straight away. One writes, recording a deployment event, and it always waits for approval.

DQL queries data stored in Grail, Dynatrace’s data store, and a DQL query only reads. Dynatrace also has its own AI, once called Davis CoPilot and now part of Dynatrace Intelligence, that turns a plain question into DQL and explains a DQL query in plain words. Botify can use it.

ActionWhat it doesApproval
list_problemsLists open and resolved problems in a time windowRuns straight away
get_problemReads one problem’s root-cause evidence and affected servicesRuns straight away
list_entitiesLists monitored hosts, services, containers or databasesRuns straight away
get_entityReads one host or service: its properties, links and tagsRuns straight away
get_entity_type_schemaShows the fields of a type, so filters are written correctlyRuns straight away
query_metricsReads a metric over a time windowRuns straight away
search_logsSearches logs with DQLRuns straight away
query_tracesSearches request traces with DQLRuns straight away
verify_dqlChecks a DQL query for mistakes without running itRuns straight away
davis_nl_to_dqlTurns a plain question into DQL with Dynatrace’s AIRuns straight away
davis_explain_dqlExplains a DQL query in plain words with Dynatrace’s AIRuns straight away
list_eventsLists events, including deployments and changes, in a windowRuns straight away
list_deploymentsLists deployments and changes to match a problem with a releaseRuns straight away
ingest_deployment_eventRecords a deployment event in DynatraceAlways waits for approval

What does Botify do in Splunk?

The Splunk integration has 8 actions, and all of them only read. The search action will not run without a time window and caps the number of results, so a vague question cannot turn into a costly search over all your data.

ActionWhat it doesApproval
run_spl_searchRuns an SPL search within a required time window and returns the first pageRuns straight away
get_search_resultsReads more pages of an earlier search without running it againRuns straight away
list_saved_searchesLists the saved searches the account can seeRuns straight away
run_saved_searchRuns a saved search by nameRuns straight away
list_fired_alertsLists alerts that fired in a time windowRuns straight away
list_indexesLists the indexes the account can seeRuns straight away
get_field_summaryLists the fields available in an indexRuns straight away
list_itsi_notable_eventsLists IT Service Intelligence notable events (needs an ITSI licence)Runs straight away

Why does recording a deployment event need approval?

Reading Dynatrace and Splunk cannot break anything, so those actions run without slowing your engineer down. Writing is different, even when it looks small. A deployment event becomes part of the timeline your team and Dynatrace use to match problems with releases. One on the wrong service or at the wrong time misleads every later investigation.

So Botify always asks a person to approve the exact event. The permission to write events (the events:ingest scope) is also left out of the default Dynatrace connection; you add it only if you want this action. In Splunk, the account’s role decides what Botify can reach, so give it a role that can search only the indexes Botify needs.

Example: a checkout slowdown at 14:05

Here is how one question, “Why did checkout get slow at 14:05?”, runs through both platforms. Botify picks each step from the result of the last one.

  • Dynatrace problems from 12:00 to now show an open problem on the checkout service, with the affected services and Dynatrace’s root-cause evidence.
  • Dynatrace deployments for the same window show a checkout release at 13:58.
  • A DQL trace search shows that most of the added time is in calls to the inventory service.
  • A Splunk search on the application index, from 13:30 to 14:30, finds a new connection-timeout error from the inventory client starting at 14:04.
  • Splunk ITSI shows a notable event on the inventory service at 14:06, and the matching Splunk alert fired.
  • Botify writes a short answer: the likely cause is the inventory client change in the 13:58 release, each claim linked to the query behind it, with a next step for the on-call engineer.

How do you keep AI-written DQL and SPL correct?

Language models write queries that look right but are quietly wrong: a wrong field name, the wrong index, a filter that returns nothing. Build the checks into the steps instead of hoping the model is careful.

  • Look first: list the indexes and fields in Splunk, and read the type’s fields in Dynatrace, before writing the query.
  • Check before running: test DQL with verify_dql, and ask for the plain explanation when an engineer needs to read the query.
  • Prefer trusted searches: a saved search your team already relies on beats a new SPL query written under pressure.
  • Limit everything: one time window for every call, and a result cap on every search.
  • Show the query: every finding includes the DQL or SPL behind it, so an engineer can rerun it in the platform itself.

Common questions

01

Can Botify write Splunk SPL and Dynatrace DQL?

Yes, both. For DQL it can also use Dynatrace’s own AI to turn a question into a query. Every query should have a time window and a result cap, be checked where the platform allows, and appear next to its results so engineers can review it.

02

Does Botify need write access to Dynatrace or Splunk?

Not for investigating. Everything in a normal investigation only reads. Write access is needed only to record a deployment event in Dynatrace, and that waits for approval every time.

03

Does Botify replace Davis AI or Splunk ITSI?

No. Dynatrace’s built-in AI and Splunk ITSI find problems and group events inside their own data. Botify reads what they found and matches it with the other platform, ServiceNow changes and past incidents.

04

What permissions should the Dynatrace and Splunk connections have?

In Dynatrace, read access to problems, entities, metrics, events, logs and traces; add event writing only if you want deployment events recorded. In Splunk, an account whose role can search only the indexes Botify needs.

05

Can Botify read ITSI notable events without ITSI?

No. Notable events are part of Splunk IT Service Intelligence, so that action needs an ITSI licence. Without ITSI, Botify still uses fired alerts, saved searches and SPL searches.

Sources

  1. [1]Dynatrace Docs: Query with natural language
  2. [2]Dynatrace Docs: Use DQL queries
  3. [3]Dynatrace Docs: Events API v2, POST an event

Where you’ll find this in Botify

Start with one assistant on WhatsApp or your website.

Book a demo and we’ll show you an assistant answering questions like your customers’. We set up the first one with you and review the results together before you add more.