Black Friday ~40% off Growth and Pro annual plans See offers
  • Agents
  • Pricing

Incident report: what happened and what customers need to do

Created Nov 3, 2026 Updated Nov 11, 2026

Chainstack is committed to the security of our platform and the protection of our customers’ data. In keeping with that commitment to transparency, we are sharing the details of a recent security incident.

Update, November 10, 2026: We added technical details of the intrusion for security teams and researchers, including indicators of compromise. See Technical details below.

What happened

Our findings indicate that this incident was part of a broader AI-powered attack campaign reported in late October. That does not change our responsibility for what happened in our environment.

An external attacker gained unauthorized access to part of Chainstack’s infrastructure through a third-party component. We detected the intrusion, cut the access paths and are hardening our systems.

Customer-facing services continued to operate normally throughout the incident. Customer nodes, the console, and the API were not modified. We do not store payment method details, such as full card numbers or security codes, so they were not exposed.

What was exposed

The attacker accessed a database containing customer account information. Information that may have been exposed includes organization names, account holder names, the email addresses of your organization’s members on Chainstack, and password hashes.

The attacker also had access to systems holding node credentials. We have no evidence they were read, but we are treating them as exposed and ask customers to rotate them.

Timeline

  • October 26, 2026: The attacker gained access to a development environment
  • October 29, 2026: The attacker gained access to some parts of our production environment
  • October 31, 2026: We detected the intrusion and cut the access paths the same day
  • November 2, 2026: We finalized the investigation and closed the incident

What we ask customers to do

We are notifying all customers by email and asking them to:

  • Change the passwords, including on any other services where they reused the same password.
  • Rotate the node credentials. Self-service rotation is now available in the console. Current endpoints keep working until customers complete the rotation, allowing them to rotate credentials without downtime. See the documentation.
Screenshot 2026 11 03 At 18.06.18 logo
  • Regenerate the platform API keys if they use the platform API.
  • Watch for phishing messages. Chainstack will never ask for passwords, tokens or keys.

Technical details

This section is for security teams and researchers: how the intrusion unfolded, the indicators we observed, and what we’re changing, so others can check their environments for the same gaps. All times are UTC.

In brief

  • The attacker entered through an internet-exposed Grafana instance in a development Kubernetes cluster, deployed for testing with its default administrator credentials unchanged and without an extra layer of protection such as single sign-on or basic authentication. Grafana’s data-source proxy gave them a path to internal services, including an unauthenticated MCP (Model Context Protocol) tool server whose service account had cluster-admin rights.
  • In development they obtained machine credentials that also reached production. On October 31 they read secrets in bulk from our production cloud secrets manager, used a compromised VPN key to connect to our database network and queried production databases.
  • The attacker used ARTEX, an open-source framework that orchestrates LLM agents to automate intrusions. Its artx/artex naming markers appear throughout the attacker’s artifacts.
  • We found no evidence that a software vulnerability was exploited. The attack chained default credentials, exposed administrative functionality, missing authentication, over-broad permissions and long-lived credentials.

How the intrusion unfolded

1. Initial access: an exposed Grafana instance (October 26)

A test Grafana instance had been deployed to a development cluster in July. It was reachable from the internet without an extra layer of protection such as single sign-on or basic authentication, and its default administrator credentials had never been changed.

From 19:26 on October 26, the attacker used Grafana’s data-source proxy as a pivot into the cluster network. A Grafana administrator can create a data source with any URL, and the proxy forwards requests from the Grafana server to that URL and returns the response. After logging in with the default administrator credentials, the attacker pointed data sources at internal addresses and, over about two hours, connected to dozens of internal services, including identity, database, automation and monitoring endpoints. The proxy had no destination allowlist, and no network policy restricted where Grafana could connect.

2. Cluster admin through an unauthenticated MCP tool server (October 26–27)

At 21:01 the attacker reached an internal MCP tool server that exposes Kubernetes operations to AI agents. It required no authentication and ran with a service account bound to cluster-admin. Network flows, those permissions and the changes that followed indicate that it served as the attacker’s command executor; the individual tool calls could not be recovered.

Over the next hour and a half the attacker:

  • tested write access to an internal workflow engine, using a workflow named artx-check-* and an identity named artex-verify-*, and created administrator users in the development identity provider;
  • deployed a test web page;
  • replaced the JavaScript served by a development web application by mounting attacker-supplied files over the originals through ConfigMap volumes. The container images did not need to change.

On October 27 the same technique was applied to our staging web application and to the development Grafana. The injected loader referenced a second-stage script on silverbullet[.]icu and used the global marker window.__artx7f3a. We did not retrieve the second stage. The loader stayed in place until we shut down the development cluster on October 31. Public scans show the same host serving bootstrap.js under per-site /p/<UUID>/ paths on other websites, where it loads further scripts that post to /vs/<UUID>/collect.

3. Credential theft from automation (October 29)

Development cloud credentials and an API key for our internal BI tool were already in use from 187.15.134[.]80 on the morning of October 29.

Between 17:03 and 17:12, the attacker rewrote a workflow in our workflow-automation platform directly in the platform’s database so that it ran shell commands, triggered it with a valid webhook token, and then restored the original. The webhook calls came through Grafana. The run returned the automation pod’s environment, its Kubernetes service-account token and a machine identity for our infrastructure access gateway. Editing the database directly bypassed the platform’s own change history. The platform allowed shell execution, and the machine identity was mounted into the same pod.

4. From development to production (October 29–31)

On October 29 at 17:08, the compromised access-gateway identity was used from 18.237.52[.]142. The same source then used a second machine identity, belonging to our deployment automation, and at 17:49 listed the secrets of a production Kubernetes cluster (a list request returns their contents) before reading several individually, including cloud and infrastructure credentials. This is the production access in the timeline above.

On October 31, with a key for the cloud service account of our infrastructure-as-code automation, the attacker read secrets in bulk from our production cloud secrets manager between 01:40 and 08:31, from 187.15.134[.]45. On the same day they:

  • created two more keys for that service account, and a Kubernetes service account with its own role and long-lived token (artx-sa, artx-observer, artx-sa-token) as an independent way back in;
  • ran short-lived pods (artx-*) in production. One ran on the host network and took the node’s cloud service-account token from the instance metadata service; a minute later the token was used from 187.15.134[.]45 to list storage buckets;
  • in another production cluster, ran a privileged pod with host networking and host PID, and installed a relay as a static pod that forwarded local port 9443 to 185.242.233[.]68:39443;
  • at 10:40, re-enrolled the compromised automation machine identity with our access gateway from an attacker-created pod (artx-tbot-keepalive-*) running under the automation platform’s trusted Kubernetes service account;
  • registered rogue instances, all named artx-node-01, with the access gateway between 11:31 and 12:08. We found no evidence that the registrations were used to access resources.

5. Database access and detection (October 31)

The secrets that were read included the private key of a VPN peer on our database network. From 03:39 a client at 185.242.233[.]68 used that key. Because two clients presented the same peer identity, the tunnel endpoint flapped between them and a database replication pipeline failed. That failure started our investigation.

From about 07:30 the attacker connected to a production database as an administrative user from attacker-created pods (artx-mq, artx-mq2). They exfiltrated query results through pod command output in base64 chunks. Between 10:39 and 10:43, with a compromised Kubernetes token, they ran commands in a database connection-pooler pod and queried PostgreSQL’s pg_shadow view, which holds role password hashes, on one instance.

Separately, between October 29 and 31 the attacker used the BI tool’s API key to run SQL queries through the tool. They read the tool’s stored data-source connection details and settings, ran reconnaissance queries against application tables, and read a credential stored in one of them. Their attempts to read server files and to gain code execution through the database failed.

6. Persistence and attempts to keep access (October 30 – November 2)

  • Development hosts. A cron job ran chisel as a reverse SOCKS tunnel to 185.242.233[.]68:10800, and a systemd timer (artx-guard) beaconed to 185.242.233[.]68:8088/artx-beacon. Both were installed on October 30.
  • Deployment repository. At 15:54 on October 31, using a compromised credential of our infrastructure automation, the attacker committed two Kubernetes Job manifests (artx-cleanup) carrying database credentials and query payloads to an internal deployment repository. The deployment controller’s sync failed because its cluster access had already been cut.
  • CI. A rogue CI runner was registered in development on October 30. We found no evidence that it received secrets or ran jobs.
  • DNS and edge. With compromised API tokens and cloud credentials for our DNS and edge providers, the attacker enumerated zones, downloaded the source of our edge scripts, and created and deleted temporary proof records (_artx-*) on our domains to test write access.

The access paths into our environments were cut on October 31, but some credentials the attacker had copied, such as API tokens for external services, kept working until we rotated them; the last use we observed was at 06:28 on November 2. That morning, two development hosts that were briefly brought back up re-ran their implants. Their outbound connections were blocked.

Detailed timeline

Time (UTC)Event
July 2026Test Grafana instance deployed to a development cluster with default administrator credentials and with no single sign-on or basic authentication in front
Oct 26, 19:26First use of Grafana’s data-source proxy to reach internal services
Oct 26, 21:01First connection to the unauthenticated MCP tool server
Oct 26, about 22:11JavaScript loader injected into a development web application
Oct 27Loader added to the staging web application and the development Grafana
Oct 29, 09:56First observed use of compromised development cloud credentials
Oct 29, 17:03–17:12Automation workflow hijacked; access-gateway machine identity compromised
Oct 29, 17:49Secrets read from a production Kubernetes cluster
Oct 30, 02:53–03:51DNS and edge API tokens used; BI tool used to read stored connection details
Oct 30, 14:10–14:37Persistence installed on development hosts
Oct 31, 01:40–08:31Bulk reads from the production secrets manager; new keys and identities created
Oct 31, 03:39–06:59Compromised VPN key disrupts database replication; intrusion detected
Oct 31, 07:29–12:24Database queries from attacker pods; metadata token theft; privileged pods and relay; rogue enrollments
Oct 31Containment: attacker sources blocked, compromised identities disabled and locked, keys deleted
Oct 31, 15:54Malicious commit to a deployment repository; deployment fails because cluster access was already cut
Nov 2, 06:28Last observed use of a copied credential

Attacker tooling and attribution

We assess with high confidence that the attacker used ARTEX:

  • Naming markers. Attacker-created resources consistently carry artx/artex prefixes, often with numeric suffixes such as artex-i37257-*. Google Threat Intelligence reported ARTEX creating artex-probe-* files with five-digit, roughly incremental numbers, and CrowdStrike lists artx/artex strings among ARTEX markers. S-RM reports ARTEX tagging artifacts with incrementing i<number> counters, the form seen in artex-i37257.
  • Tools. CrowdStrike also reports ARTEX operators tunneling through chisel, which this attacker used for persistence.
  • Behavior. Many attack paths ran in parallel, with rapid retries, and with test writes and throwaway DNS records used only to check capabilities.

The link between this intrusion and the broader AI-powered attack campaign mentioned above is the tooling: the same framework, the same naming markers and some of the same techniques. Public reporting ties ARTEX to several unrelated groups, so this does not identify who was behind this intrusion or show that the same operators were involved. None of our indicators appear among the actor-specific indicators published by CrowdStrike for Vapor Panda or by Microsoft for Storm-0940 and Storm-3358. We therefore attribute the tooling, not the actor.

How we verified customer-facing services

The statement above that customer nodes, the console and the API were not modified rests on:

  • Change attribution. Using our access gateway’s audit log, the clusters’ API audit logs and our infrastructure-as-code apply records, we attributed every change made through the Kubernetes API of our production clusters since October 29, including ConfigMap and workload changes, to our staff, CI, infrastructure automation or cluster controllers. The only exceptions are the attacker-created objects listed in this section, none of which changed customer node, console or API workloads.
  • Images. The images running the console and API match their registry digests, and images pushed during the intrusion window were deployed only through reviewed changes.
  • Charts and routing. No published deployment chart was overwritten, and cluster ingress and routing configuration changed only through our own deployments.
  • Web assets. No attacker change in production used the ConfigMap technique seen in development and staging, and a review of the JavaScript served by our production web applications found no injected code.

Indicators of compromise

These are point-in-time observations. The addresses belong to hosting and VPN providers and may be reused by others, so use them for time-bounded hunting rather than permanent blocking.

Network

IndicatorTypeObserved (UTC)Network (ASN)Activity
18.237.52[.]142IPv4Oct 29, 17:08–17:52Amazon (AS16509)Used compromised access-gateway machine identities; read production secrets
187.15.134[.]80IPv4Oct 29, 09:56–11:51GSL Networks (AS137409)Used compromised development cloud credentials and the BI tool’s API key; DNS writes
187.15.134[.]77IPv4Oct 30, 02:53 – Oct 31, 16:59GSL Networks (AS137409)DNS and edge token use, development cluster access, BI tool access, credential probing
187.15.134[.]45IPv4Oct 30 – Oct 31, 22:39GSL Networks (AS137409)Production secret reads, key creation, Kubernetes actions, BI tool access, DNS writes, use of the node metadata token
185.242.233[.]68IPv4Oct 30 – Nov 2, 06:28Cloudie Limited (AS55933)VPN peer impersonation; later use of copied credentials; destination of the tunnel, beacon and relay below
185.242.233[.]68:10800IP:portOct 30 – Nov 2Cloudie Limited (AS55933)chisel reverse SOCKS tunnel from development hosts
185.242.233[.]68:8088/artx-beaconIP:port/pathOct 30 – Nov 2Cloudie Limited (AS55933)Beacon from the artx-guard systemd timer
185.242.233[.]68:9331IP:portOct 31Cloudie Limited (AS55933)Served a VPN client binary to an attacker workload
185.242.233[.]68:39443IP:portOct 31Cloudie Limited (AS55933)Destination of the static-pod relay (local port 9443)
silverbullet[.]icuDomainOct 26–31Registered Oct 12, 2026Second-stage host referenced by the injected JavaScript loader
hxxps://silverbullet[.]icu/p/17fbad90-3094-4198-9b60-49fc852e331d/bootstrap.jsURLOct 26–31Second-stage script referenced by the loader

File hashes (SHA-256)

These are hashes of the application files the attacker modified to carry the loader. They are not hashes of the second stage or of host binaries.

SHA-256File
dce9be91f9c5fb80ffa3461a4bdae90f933431c6d9df2042bddb81710fa2c517Injected UI bundle, development web application
a5cd94ca9d819effde8c2362c21050981486752163854a502f350db6587757c5Injected index page, development web application
57b2479607e42d54c2705b747659a0ee2470ba10a8a7a12b1385723a21b251f8Injected UI bundle, staging web application
caf6aa2e8bf5585687b0ed96a041f9c4ea1a17742118a57418c2b9dc027486ceInjected index page, staging web application
07691e5949a123e313e1d589288e3831ec8ac30b90c45ab99d3b6ad07f23789aInjected Grafana application bundle
455473033f7ecbeafaa70e38f41f924377d54013efedeb0996ee84d3422c10ceInjected Grafana runtime bundle
7c5925a3bc97983bfbfccdc514edb4dadbdb5b796a0747a764b0dcc4f103ea15Attacker test page, app.js
b4a701949a1489e81754637238ed3f8177520f9f950e655f9357900d9e30aea8Attacker test page, index.html

Host, Kubernetes and application artifacts

  • Files: /etc/cron.d/artx-ngx, /etc/artx-ngx-node.sh, /usr/local/bin/artx-ngx-chisel
  • systemd units: artx-guard.timer, artx-guard.service, and a variant named artx-guard-gke
  • Kubernetes objects: service account artx-sa, token secret artx-sa-token, role and role binding artx-observer, and pods artx-vant, artx-vant-mb, artx-mq, artx-mq2, artx-map, artx-nr2, artx-j, artx-relay (a socat binary was dropped in it), artx-p2, rt-agent, rt-node-probe, artx-tbot-keepalive-*
  • Job manifests named artx-cleanup, committed to a deployment repository
  • ConfigMaps named artex-i<5 digits>-*, mounted over application files to replace served JavaScript
  • Hostname presented during access-gateway enrollment: artx-node-01
  • Markers: JavaScript global window.__artx7f3a; workflow and identity names artx-check-* and artex-verify-*; DNS proof-record labels _artx-* (for example _artx-probe-<digits>, _artx-dnsproof)

Names alone are weak detections. Correlate them with the timing and behavior described above.

MITRE ATT&CK mapping

Mapped to ATT&CK Enterprise v20.

TacticTechniqueHow it appeared
Resource DevelopmentT1588.007 Obtain Capabilities: Artificial IntelligenceARTEX agentic framework
Initial AccessT1078.001 Valid Accounts: Default AccountsLogin to internet-exposed Grafana with unchanged default administrator credentials, with no single sign-on or basic authentication in front
Initial AccessT1133 External Remote ServicesVPN connection with a compromised peer key
Initial Access, PersistenceT1078.004 Valid Accounts: Cloud AccountsCompromised cloud service-account keys and machine identities
ExecutionT1609 Container Administration CommandKubernetes operations through the MCP tool server; commands run in pods
ExecutionT1059.004 Command and Scripting Interpreter: Unix ShellShell commands through a rewritten automation workflow
ExecutionT1610 Deploy ContainerShort-lived artx-* pods
PersistenceT1098.001 Account Manipulation: Additional Cloud CredentialsExtra service-account keys
PersistenceT1136 Create AccountKubernetes service account; identity-provider administrators
PersistenceT1053.003 Scheduled Task/Job: Cronchisel launched from cron
PersistenceT1053.006 Scheduled Task/Job: Systemd Timersartx-guard timer
Privilege EscalationT1611 Escape to HostPrivileged pods with host network and host PID; a static pod installed on a node
Defense EvasionT1070 Indicator RemovalHijacked workflow restored after use
Credential AccessT1552.007 Unsecured Credentials: Container APIKubernetes secrets listed and read
Credential AccessT1555.006 Credentials from Password Stores: Cloud Secrets Management StoresBulk reads from the cloud secrets manager
Credential AccessT1552.005 Unsecured Credentials: Cloud Instance Metadata APINode service-account token taken from instance metadata
DiscoveryT1046 Network Service DiscoverySweep of internal services through the Grafana proxy
CollectionT1213 Data from Information RepositoriesSQL run through the BI tool
Command and ControlT1090.001 Proxy: Internal ProxyGrafana data-source proxy used to reach internal services
Command and ControlT1572 Protocol Tunnelingchisel reverse SOCKS tunnel
Command and ControlT1105 Ingress Tool TransferVPN client and socat binaries brought into workloads
ExfiltrationT1041 Exfiltration Over C2 ChannelQuery output carried out in base64 chunks through pod command output

Detection opportunities

If you run similar stacks, these checks would have surfaced this activity earlier:

  • Grafana: data-source proxy requests (/api/datasources/proxy/) to cluster-internal addresses, and new or changed data sources.
  • MCP and other AI tool servers: endpoints that accept unauthenticated requests, and service accounts with broad permissions.
  • Kubernetes: ConfigMap volumes mounted over application files outside your deployment pipeline; static pods not in your baseline; pods with host network or host PID; secret list requests from automation identities; objects named artx-* or artex-*.
  • Machine-identity enrollment: new instances or bots joining your access gateway from unexpected workloads or hostnames.
  • Cloud: bursts of secret reads by one principal; new user-managed service-account keys; node metadata tokens used from outside your network.
  • VPN: the same peer key used from two endpoints.
  • DNS: short-lived records you didn’t create, especially with an _artx- prefix.
  • Automation: workflow definitions that change without a matching entry in the platform’s own history.

What we did

  • Cut the attacker’s access: blocked their network sources; disabled compromised cloud identities and deleted their keys; locked compromised access-gateway identities and removed their enrollment rules; removed attacker-created Kubernetes identities, pods and the relay; revoked long-lived Kubernetes tokens; restricted access to Kubernetes API endpoints; and shut down the affected development cluster.
  • Verified that customer nodes, the console and the API were not modified (see How we verified customer-facing services above).
  • Stopped and disabled the persistence on development hosts.
  • Rotated our exposed credentials and renewed cluster certificate authorities and join tokens.
  • Invalidated customer login sessions.

What we’re changing

Our remediation program is working toward these outcomes. Some are in place; others are in progress.

  1. No administrative tools on the internet without an extra authentication layer such as SSO. Test environments change only through reviewed configuration, and we reconcile what is deployed against the repository. Default credentials are replaced at deployment.
  2. Services that make requests on a user’s behalf are treated as privileged. These include data-source proxies such as Grafana’s, webhooks and connectors. They get destination allowlists, and network policies limit what they can reach. For Grafana, that means setting data_source_proxy_whitelist to the hosts its data sources use and allowing outbound connections only to those data sources.
  3. Internal services that expose privileged operations require authentication and least privilege, wherever they run. That includes AI tool servers such as MCP servers: none holds cluster-admin rights, and each is isolated at the network layer.
  4. Platforms that run user-defined workflows or code don’t hold reusable identities. They get no shell execution, and short-lived, narrowly scoped credentials instead of mounted machine identities.
  5. Development has no path to production. Each environment has its own identities and secret stores, and no production secrets are copied into development.
  6. Fewer long-lived keys. Workload identity replaces static keys where possible, pods can’t reach instance metadata, and broad node service-account roles are removed.
  7. Admission control for privileged pods. Host network, host PID and privileged containers need an approved exception.
  8. No secrets in analytics replication. Credential-bearing tables and columns are excluded from data replicated into analytics systems.
  9. Rotation in dependency order. Every readable copy of a secret is closed before the secret is replaced.
  10. Logging that answers “who read what”. Data-access logging for storage, secrets and databases, kept off-host, with alerts for bulk secret reads, new service-account keys and metadata tokens used from outside.

If you see these indicators in your environment, have related findings or would like more detail, contact [email protected].

This page is the single source of truth for the incident. We will update it if new information becomes available.

If you have any further questions, contact us.

SHARE THIS ARTICLE
Customer Stories

DIA

Handling large volumes of data with a reliable websocket implementation

Darkpool Liquidity

Develop on various networks and protocols with ease, expanding at scale in a short period of time.

Coin98

Resolving performance bottlenecks native swaps, token transfers, balance loading and on-chain tracking.