Introduction

AccuKnox is thrilled to announce the availability of its runtime observability tool - AccuKnox Insights. This provides observation of workloads (k8s pods, processes in VMs) at runtime. This delivers deep visibility into the workloads and their behavior with respect to the host environment and other services.

AccuKnox insights is part of a CLI (and API) tool that provides runtime visibility in an aggregated form (by pods, processes, workloads)

  • Network: The L3, L4 and L7 connections with eBPF based observability. Ingress and egress..
  • System: The files accessed, processes forked, network connections and capabilities requested

This telemetry is derived from observing system calls that processes make while interacting with the host environment.

Solving the runtime security problem


AccuKnox insights helps you to understand what exactly is happening to your workload at runtime, after static security checking and controls (image scanning, patching, configuration checks and after admission control). Posture management tools are increasingly applied to secure cloud workloads. But,  these static checks are blind to unknown (zeroday) vulnerabilities, unintended misconfigurations and coding bugs, etc.

AccuKnox provides easy to understand visibility  into:

  1. What network connections have been established and across what ports; what connections were denied
  2. What files were accessed, directories and how many times
  3. What process forking is happening and by how many times
  4. What processes have even attempted network connections?

Many of these events can detect anomalous behavior as identified by the MITRE ATT&CK Framework. Examples could be a java microservice that has been compromised through the Log4J vulnerability and is attempting to make an external network connection.

AccuKnox insights provides anomalous behavior as an aggregated log so that the user can take an action and remediate the same.

AccuKnox insights works by tapping into the system calls that each workload does while interacting with the host operating system. By collecting the series of events that occur as the workloads interact with the host operating system and other workloads.

AccuKnox Insights

AccuKnox insights allows you to get an aggregated view of the key events at runtime as described above and also allows you to filter based on specific events or field types. AccuKnox insights in turn leverages Cilium and KubeArmor for observability

Network specific features: AccuKnox insights provides many network specific fields that are summarized to reduce verbosity.

For networks, AccuKnox insights can identify and aggregate

  • All network (L3, L4, and L7) connections from a particular workload (inbound / outbound) and whether it was allowed by the policies or not.

For KubeArmor, AccuKnox can identify a summary of

  • processes getting forked
  • directories and files getting accessed
  • network connections being attempted

In the following example, we see a workload with various pods making ingress and egress (these are summaries and therefore unique events) with various source and destination IP addresses.

shell>accuknox-cli observe -network


SOURCE POD NAME                  SOURCE IP   DESTINATION IP  TRAFFIC DIRECTION  PROTOCOL   STATUS    TAG
knoxautopolicy-6fbf6f6c76-mz87p      10.0.0.120      10.0.0.204   INGRESS          UDP    ALLOW     MITRE
coredns-96cc4f57d-tc7fr          10.0.0.204      10.0.0.120  EGRESS    UDP      ALLOW     MITRE
knoxautopolicy-6fbf6f6c76-mz87p      10.0.0.120      10.0.0.204  INGRESS   UDP   ALLOW     MITRE
coredns-96cc4f57d-tc7fr          10.0.0.204      10.0.0.120  EGRESS    UDP   ALLOW    MITRE
Mysql-0            10.0.0.120      10.0.0.204  INGRESS    TCP   DENY       MITRE
coredns-96cc4f57d-tc7fr          10.0.0.204      10.0.0.120  EGRESS     UDP    ALLOW    MITRE
knoxautopolicy-6fbf6f6c76-mz87p      10.0.0.120      10.0.0.204  INGRESS    UDP   ALLOW    MITRE
coredns-96cc4f57d-tc7fr          10.0.0.204      10.0.0.120  EGRESS      UDP    ALLOW   PCI-DSS
knoxautopolicy-6fbf6f6c76-mz87p      10.0.0.120      10.0.0.204  INGRESS    UDP   ALLOW   PCI-DSS
coredns-96cc4f57d-tc7fr          10.0.0.204      10.0.0.120  EGRESS      UDP    ALLOW   MITRE

The pod traffic might have been allowed or denied. And if there’s a policy attached that is explicitly denying the pod, the tag of the policy is shown to the right.

shell>accuknox-cli observe -system


POD-NAME           OPERATION    RESOURCE                          LAST_UPDATED           STATUS    TAG
checkoutservice-8f859666-hm4jj         Network    domain=AF_INET type=SOCK_DGRAM|SOCK_NONBLOCK|SOCK_CLOEXEC protocol=0   2022-04-26 15:59:41 +0530 IST  Passed    MITRE
checkoutservice-8f859666-hm4jj          Network    sa_family=AF_INET sin_port=53 sin_addr=10.43.0.10                      2022-04-26 15:59:41 +0530 IST  Passed    MITRE
checkoutservice-8f859666-hm4jj          Network    sa_family=AF_INET sin_port=53 sin_addr=10.43.0.10                      2022-04-26 15:59:41 +0530 IST  DENY    MITRE
recommendationservice-77bdd78d96-zj544  File       /sys/kernel/mm/transparent_hugepage/hpage_pmd_size                     2022-04-26 15:59:40 +0530 IST  Passed    PCI-DSS
recommendationservice-77bdd78d96-zj544  Process    /bin/grpc_health_probe -addr=:8080                                     2022-04-26 15:59:40 +0530 IST  Passed    MITRE
recommendationservice-77bdd78d96-zj544  File       /etc/localtime                                                         2022-04-26 15:59:40 +0530 IST  DENY    PCI-DSS
recommendationservice-77bdd78d96-zj544  Network    domain=AF_INET type=SOCK_STREAM|SOCK_NONBLOCK|SOCK_CLOEXEC protocol=0  2022-04-26 15:59:40 +0530 IST  Passed    MITRE
cartservice-5dff477f54-tv78x            File       /sys/kernel/mm/transparent_hugepage/hpage_pmd_size                     2022-04-26 15:59:40 +0530 IST  Passed    MITRE
cartservice-5dff477f54-tv78x            File       /                                                                      2022-04-26 15:59:40 +0530 IST  Passed    PCI-DSS
cartservice-5dff477f54-tv78x            File       /sys/kernel/mm/hugepages                                               2022-04-26 15:59:40 +0530 IST  Passed    MITRE

In the above example, the KubeArmor observed insights are shown in an aggregated view, showing the host interaction of the pod including files opened, processes forked etc.  The operation is indicated under the operation field along with last observed, the resource used as well as whether the operation was allowed (or denied) due to a security policy implemented in KubeArmor.

Please keep in mind that these are unique summaries (without repetition). The tag for which the policy was denied is also mentioned to the right.

Demonstrating an attack and observing it with insights:

A common use case is to prevent untrusted shell access within pods. Let’s apply the following KubeArmor policy to this workload:

# To learn more about KubeArmor visit: 
# https://www.accuknox.com/kubearmor/ 

apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: ksp-block-untrusted-shell-execution
  namespace: default # Change your namespace
spec:
  tags : ["MITRE","D3fend","Execution","Unix Shell"] 
  message: "Bash shells have been accessed"
  selector:
    matchLabels:
      app: testpod 
  process:
    severity: 2 # Higher severity for processes 
    matchPaths:
    - path: /bin/bash
    - path: /bin/sh
    - path: /usr/bin/bash
    - path: /usr/bin/env
    - path: /usr/bin/shell
    - path: /bin/ksh
    - path: /etc/init.d      
    - path: /dev/tty 
    - path: /bin/zsh
    - path: /bin/tcsh
    - path: /bin/csh
    action: Block   
  file:
    severity: 10  # lowest severity for processes invoked as child process of bash
    matchPaths:
    - path: /bin/bash
    - path: /bin/sh
    - path: /usr/bin/bash
    - path: /usr/bin/env
    - path: /usr/bin/shell
    - path: /bin/ksh
    - path: /etc/init.d      
    - path: /dev/tty 
    - path: /bin/zsh
    - path: /bin/tcsh
    - path: /bin/csh
      fromSource:
      - path: /bin/bash
    action: Block

You’ll need to replace the pod labels with your current pod label to ensure that the policy is applied to the correct pod.

The Kubearmor policy can then be applied with the following CLI command:

Kubectl -f policy-name.yaml

Now let’s try to exec into the pod by using the following command

kubectl exec -it <pod_name> -- /bin/bash</pod_name>

The above policy will  deny the attempt and generate a log.

shell>accuknox-cli observe -system

POD-NAME           OPERATION    RESOURCE                          LAST_UPDATED           STATUS    TAG

recommendationservice-77bdd78d96-zj544  Process    /bin/bash                                   2022-04-26 15:59:40 +0530 IST  DENY    MITRE

Summary

This is just one example of how you can easily use AccuKnox’s Insights to observe runtime security violations. Stay tuned to this blog for more examples, or contact us with any questions at [email protected].