All the library’s versions between 2.0 and 2.14.1 included are affected.
JDKs versions higher than 6u211, 7u201, 8u191 are not affected by the LDAP RCE attack vector, as the
com.sun.jndi.ldap.object.trustURLCodebase is disabled by default, hence JNDI cannot load a remote codebase using LDAP.
In releases prior to the version 2.10, this behavior can be mitigated by setting either the system property log4j2.formatMsgNoLookups.
The environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS to true.
For releases from 2.0-beta 9 to 2.10.0, the mitigation is to remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class.
How does this work?
Log4j doesn't treat strings as just strings. It inspects its contents and checks if it contains any "variables" that needs resolution including objects that needs lookup.
This include Jndi lookups apart from a range of other lookups. The JndiLookup allows variables to be retrieved via JNDI.
There are other lookups are available as well as part of the specification. The full flow looks like this:
The steps of this attack are as follows:
- Log4j receives a malicious message with a JNDI lookup
- The JNDI lookup uses LDAP protocol to Fetch the object. The object fetch esseentially is Initialcontext.lookup() that loads the object into the memory space of the calling process.
- A malicious java class object is then loaded into the memory of the calling process leading to Remote Code Execution (RCE). This is not a process fork yet as the object is within the memory space.
- The Remote code i.e. the binary object loaded could then propogate further attacks including access to file path, additonal binary downloads and execution of the same. Other Tactics, techniques and procedures (TTPs) as documented by MITRE become applicable
The attack is carried out by performing a JNDI lookup internally by either using LDAP or RMI protocol. The lookup is performed by a malicious request,
In previous releases (>2.10) this behavior can be mitigated by setting system property "log4j2.formatMsgNoLookups" to "true" or by removing the JndiLookup class from the classpath (example: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class).
Java 8u121 (see https://www.oracle.com/java/technologies/javase/8u121-relnotes.html) protects against remote code execution by defaulting "com.sun.jndi.rmi.object.trustURLCodebase" and "com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".
What is actually happening is that Log4J attempts to resolve and looks up objects (this is by design and not a bug) as a part of its logging framework. Additionally, total lack of application hardening and run-time security of the application essentially removes any other guardrails and provides the application to execute any arbitrary remote code.
How to Permamently fix Log4J vulnerability
There are a couple of things that needs to get done while attempting to fix this issue for good from appearing in any part of the workload.
A good fix should attempt to do the following:
- #1: Block execs from the JVM and most other Tactics, Techniques and Procedures (TTPS) that exploits like this use. This can be done by implementing policies that can enforce MITRE tactiques. Please visit https://attack.mitre.org/# for more info.
- #2: Block RMI / LDAP lookups - As much as many vendors will tell you, completely blocking the RCE for an in-process / in-memory java class object is impossible without actually blocking the RMI / LDAP call itself, which brings us to the next step which is to completely block RMI and LDAP.
Preventing execs from the JVM/Java
Note: The policies below require KubeArmor (https://github.com/kubearmor) and Accuknox's version of Cilium (https://github.com/kubearmor/cilium). For more information on installing a sample application with KubeArmor and Cilium - please refer Accuknox's open source quick start guide: https://help.accuknox.com/open-source/quick_start_guide/
An attacker infiltrates with the intent to either exfiltrate the internal data or for cryptomining or to simply wreck havoc in the internal apps with the intent to make it unavailable. In all these cases, the attacker needs to execute an arbitrary program that can fulfill its malicious intent. This vulnerability allows the attacker to place a binary within the internal network. However, guardrails can be placed so as not to allow the JVM to spawn processes.
apiVersion: security.kubearmor.com/v1 kind: KubeArmorPolicy metadata: name: do-not-allow-exec-from-java spec: severity: high message: "disallow execing from java process" selector: matchLabels: app: log4j2 process: matchPaths: - path: * #disaallow all paths from the java process fromSource: - path: /opt/openjdk-16/bin/java action: Block
Additional MITRE based TTP's have been documented as policy templates at https://github.com/kubearmor/policy-templates
Blocking RMI and LDAP Lookups
Please note: The policy below only works for Kubernetes pods at the moment and requires Cilium
RMI (Remote Method Invocation) is a special purpose use-case and not all the applications using log4j might be utilizing that feature. If not in use, it is advised to block the access to the RMI port, thus ensuring that the attacker cannot use an unused port for malicious intent.
Following is a CiliumNetworkPolicy that can block the access to the port * (this currently works in the context of Kubernetes pods).
apiVersion: "cilium.io/v2" kind: CiliumNetworkPolicy metadata: name: "Block LDAP and RMI Calls" spec: endpointSelector: matchLabels: app: log4j2 egressDeny: - fromEndpoints: toPorts: - ports: - port: "389" protocol: ANY - port: "636" protocol: ANY - port: "1099" protocol: ANY
Arbitrary Code Execution and Zero Trust
The obvious next question is how to prevent possibility of misuse of such vulnerabilities in the future.
Arbitrary Code Execution is a major attack mode and one has to focus on defining what makes the code “arbitrary”. Arbitrary in the context could be defined as anything that is not in the usual execution context.
Using Zero-trust (ZTNA) architecture requires one to specify a least-permissive policy set that only allows whitelisted actions and deny everything else. Thus, having a Zero-Trust posture could effectively guard an organization from the possibilities of such attacks.
However, achieving Zero-Trust in practice is much more challenging. Zero-trust requires that an organization have appropriate automation, software deployment processes coupled with the right tools. Few points to ponder over could be:
- Having flexible policy enforcement engines is not good enough. How to achieve a least permissive policy set that goes with those policy engines?
- If a developer makes changes to the app, do you have an automated process to inculcate new rules that might have changed due to app changes?
- Does the org have a flexible EDR/XDR that allows the DevSecOps and Security teams to focus on right events?
How could a Zero Trust posture prevent misuse of log4j vulnerability?
A Zero-Trust posture across network and applications/systems could be defined as follows:
- Only allow ingress/egress connections the application is supposed to make/handle.
- Only allow the process execs that are in the allowed list.
- Only allow file-system path accesses that the application needs.
- Only allow the system capabilities that are required for application’s normal needs.
Achieving this posture is easier said than done.
Accuknox and Zero Trust
Accuknox provides a flexible policy enforcement engines coupled with the right policy discovery/recommendations tools that precisely help an organization answer the above questions. Accuknox has built the policy engines with basic design tenets in mind i.e., every policy engine must support observability, auditing (dry-run), and enforcement options.
Observability coupled with policy discovery engine can provide an organization with the least-permissive policy settings required.
Example observability and enforcement logs from Accuknox policy engines
Example observability log when a execve is used by the app:
== Log / 2021-12-12 19:48:37.737160 == Cluster Name: Default Host Name: pandora Namespace Name: default Pod Name: log4j-kubearmor Container ID: 7ccca0b0a09ba86c96d581f695a534d0ec1d7a844f30efc16e0e72568a84cc39 Container Name: log4j-kubearmor Type: ContainerLog Source: jspawnhelper Operation: Process Resource: /bin/touch /tmp/log4jServerp0wn3d Data: syscall=SYS_EXECVE Result: Passed
Blocking any processes spawned from JVM/Java process results in following alert while the execve is denied (Note that KubeArmor is an enforcement engine):
== Alert / 2021-12-12 19:57:07.871126 == Cluster Name: Default Host Name: pandora Namespace Name: default Pod Name: log4j-kubearmor Container ID: 7ccca0b0a09ba86c96d581f695a534d0ec1d7a844f30efc16e0e72568a84cc39 Container Name: log4j-kubearmor Policy Name: do-not-allow-exec-from-java Severity: 5 Message: disallowed execing from java process Type: MatchedPolicy Source: jspawnhelper Operation: Process Resource: /bin/touch /tmp/log4jServerp0wn3d Data: syscall=SYS_EXECVE Action: Block Result: Passed
Questions.. Need help?
Feel free to reach out to us on support [at] accuknox.com for any support or help with your Log4J issues.
Now you can protect your workloads in minutes using AccuKnox, it is available to protect your Kubernetes and other cloud workloads using Kernel Native Primitives such as AppArmor, SELinux, and eBPF.
Let us know if you are seeking additional guidance in planning your cloud security program.
Read more blogs from Cloud Security Category here.