Use JMX to Change the Log level dynamically
In this note, we will create a program that can change the log level dynamically on the local or remote server. We will use Spring JMX exporter. Log4j is used. The applicaiton server is Weblogic 10.3.2. On the remote side, the application is deployed to a cluster of two weblogic managed servers weblogic01 and weblogic02. Note that on later versions of Weblogic such as version 11, the MBean server configurations may be different.
There are several files needed for this.
What are needed on the application side
1. In the application, create the following classes:(1) IAppLogUtils.java
package com.yourcompany.app.server.utils;
public interface IAppLogUtils {
public void updateAllLogLevels(String s);
}
(2) AppLogUtils.java
package com.yourcompany.app.server.utils;
import java.util.Enumeration;
import org.apache.log4j.Level;
import org.apache.log4j.LogManager;
import org.apache.log4j.Logger;
public class AppLogUtils implements IAppLogUtils {
public void updateAllLogLevels(String s) {
Enumeration enumeration = LogManager.getCurrentLoggers();
do {
if (!enumeration.hasMoreElements())
break;
Logger logger = (Logger) enumeration.nextElement();
logger.setLevel(Level.toLevel(s));
} while (true);
}
}
(3) In the application context xml file, add the following to use the Spring JMX exporter.
<bean id="jmxServerRuntime" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jmx/runtime"/>
</bean>
<bean id="appLogUtils" class="com.yourcompany.app.server.utils.AppLogUtils" />
</bean>
<bean id="exporter" class="org.springframework.jmx.export.MBeanExporter">
<property name="beans">
<map>
<entry key="APP:name=appLogUtils" value-ref="appLogUtils" />
</map>
</property>
<property name="registrationBehaviorName" value="REGISTRATION_IGNORE_EXISTING"/>
<property name="server" ref="jmxServerRuntime"/>
</bean>
Two important notes here.
(a) A JVM can have multiple MBean servers. We will register our JMX bean on the weblogic runtime mbean server. Later on when we create the JMX client program to change the log level, we will connect to this mbean server. A test shows that if we do not specify the server property as in the above, Spring MBeanExporter will export the MBean to the JVM platform mbean server. On local machine, the jconsole program will detect this JVM platform mbean server. It does not need to have the username/password to access it.
(b) The value for the property "registrationBehaviorName" of the MBeanExporter is set. The default value is REGISTRATION_FAIL_ON_EXISTING, which will give error if Spring registers the Mbean with the same name twice. The error message is like this: "javax.management.InstanceAlreadyExistsException".
What are needed on the client side to invoke the JMX bean
It is relatively easy if the weblogic server is running locally. If the servers are remote, the program is still similar but a little bit more complex. The following program has code for both cases.
In the program, we assume that the two managed weblogic servers have the following host name and port number:
host1 14110 host2 14110
Note that the port numbers are the same because they are in the same cluster of weblogic managed servers. The weblogic admin server has a different port number. You can also connect to the admin server to get its JMX information. You need to use the username/password to connect to the admin server. Interestingly, you need to use the same username/password to connect to the managed weblogic servers for the JMX connection.
import java.io.IOException;
import java.util.Hashtable;
import java.util.Set;
import javax.management.MBeanServerConnection;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;
import com.bea.core.repackaged.springframework.util.Assert;
public class WeblogicJMXClient {
// this method changes the log level to "DEBUG" on the locally running weblogic server.
public void testLocT3() throws Exception {
JMXConnector jmxCon = null;
try {
JMXServiceURL serviceUrl = new JMXServiceURL("t3", "localhost",
7001, "/jndi/weblogic.management.mbeanservers.runtime");
System.out.println("Connecting to: " + serviceUrl);
Hashtable env = new Hashtable();
env.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,
"weblogic.management.remote");
env.put(javax.naming.Context.SECURITY_PRINCIPAL, "username");
env.put(javax.naming.Context.SECURITY_CREDENTIALS, "password");
jmxCon = JMXConnectorFactory.newJMXConnector(serviceUrl, env);
jmxCon.connect();
MBeanServerConnection con = jmxCon.getMBeanServerConnection();
Set<ObjectName> mbeans = con.queryNames(new ObjectName(
"APP:name=appLogUtils"), null);
ObjectName logService = (ObjectName) mbeans.iterator().next();
Object result = con.invoke(logService, "updateAllLogLevels",
new Object[] { "DEBUG" }, new String[] { String.class
.getName() });
} finally {
if (jmxCon != null)
jmxCon.close();
}
}
/**
* This method changes the log level to the specified value "INFO" or "DEBUG" on the remote
* weblogic server cluster
*/
public void changDevLog(String logLevel) {
Assert.isTrue("DEBUG".equals(logLevel) || "INFO".equals("logLevel"));
JMXConnector jmxCon = null;
// change on weblogic01
try {
jmxCon = getJMXConnector("t3", "host1",
14110, "username", "password");
changeLogLevel(jmxCon, logLevel);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (jmxCon != null) {
try {
jmxCon.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
// change on weblogic02
try {
jmxCon = getJMXConnector("t3", "host2",
14110, "username", "password");
changeLogLevel(jmxCon, logLevel);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (jmxCon != null) {
try {
jmxCon.close();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
}
private JMXConnector getJMXConnector(String protocol, String host,
int port, String username, String password) throws IOException {
JMXConnector jmxCon = null;
JMXServiceURL serviceUrl = new JMXServiceURL(protocol, host, port,
"/jndi/weblogic.management.mbeanservers.runtime");
System.out.println("Connecting to: " + serviceUrl);
Hashtable env = new Hashtable();
env.put(JMXConnectorFactory.PROTOCOL_PROVIDER_PACKAGES,
"weblogic.management.remote");
env.put(javax.naming.Context.SECURITY_PRINCIPAL, username);
env.put(javax.naming.Context.SECURITY_CREDENTIALS, password);
jmxCon = JMXConnectorFactory.newJMXConnector(serviceUrl, env);
return jmxCon;
}
private void changeLogLevel(JMXConnector jmxCon, String level)
throws Exception {
jmxCon.connect();
MBeanServerConnection con = jmxCon.getMBeanServerConnection();
Set<ObjectName> mbeans = con.queryNames(new ObjectName(
"APP:name=appLogUtils"), null);
ObjectName logService = (ObjectName) mbeans.iterator().next();
Object result = con
.invoke(logService, "updateAllLogLevels",
new Object[] { level }, new String[] { String.class
.getName() });
}
public static void main(String[] args) throws Exception {
WeblogicJMXClient c = new WeblogicJMXClient();
//c.testLocT3();
c.changDevLog("DEBUG");
}
}
MBean Server on Weblogic
The following is about Weblogic 10.
At the core of any JMX agent is the MBean server, which acts as a container for MBeans.
In Weblogic, the JVM for an Administration Server maintains three MBean servers provided by BEA and optionally maintains the platform MBean server, which is provided by the JDK itself. The JVM for a Managed Server maintains only one BEA MBean server and the optional platform MBean server.
The three MBean servers provided by BEA are:
- Domain Runtime MBean Server. Only the Administration Server hosts an instance of this MBean server. Its INDI name is "weblogic.management.mbeanservers.domainruntime".
- Runtime MBean Server. Each server in the domain hosts an instance of this MBean server. Its JNDI name is "weblogic.management.mbeanservers.runtime".
- Edit MBean Server. Only the Administration Server hosts an instance of this MBean server. Its JNDI name is "weblogic.management.mbeanservers.edit".
The JVM's platform MBean server is provided by the JDK that contain monitoring information for the JVM itself. You can register custom MBeans in this MBean server, but BEA recommends that you register them in its Runtime MBean Server.
You can also configure the WebLogic Server Runtime MBean Server to be the platform MBean server, in which case the platform MBean server provides access to JVM MBeans, Runtime MBeans, and active configuration MBeans that are on a single server instance.
Register an MBean on Weblogic
Local clients can access a WebLogic Server instance’s Runtime MBean Server through the JNDI tree instead of constructing a JMXServiceURL object. Only the Runtime MBean Server registers itself in the JNDI tree.
If the classes for the JMX client are located at the top level of an enterprise application (that is, if they are deployed from the application’s APP-INF directory), then the JNDI name for the Runtime MBean Server is: java:comp/jmx/runtime
If the classes for the JMX client are located in a Java EE module, such as an EJB or Web application, then the JNDI name for the Runtime MBeanServer is: java:comp/env/jmx/runtime
In our case for the bean AppLogUtils, we use the JNDI name "java:comp/env/jmx/runtime". This bean AppLogUtils is a spring bean and is used in a web module deployed on the Weblogic server. So according to the above stated, this bean is registered at the Weblogic runtime MBean server. And in our subsequent code we do see that the JMX client connects to the runtime MBean server to manage this bean because it uses the JNDI name "/jndi/weblogic.management.mbeanservers.runtime".
One thing that may be misleading is the JNDI name "java:comp/env/jmx/runtime". This name looks like a java standard since it does not contain any thing about Weblogic. But it may just be a proprietary name used by Weblogic for the java platform MBean server. As stated before, all the three Weblogic MBean servers ( Domain Runtime MBean server, Runtime MBean server, Edit MBean server ) have their JNDI name and the client can connect to them programmatically using these JNDI names remotely. But there does not seem to be a JNDI name for the platform MBean server for the remote client to connect to. The JNDI name "java:comp/env/jmx/runtime" is only used locally. The following is from Oracle document about how to access the JVM platform MBean server remotely:
"Remote access to the platform MBean server can be secured only by standard JDK 1.5 security features (see http://java.sun.com/j2se/1.5.0/docs/guide/management/agent.html#remote). If you have configured the WebLogic Server Runtime MBean Server to be the platform MBean server, enabling remote access to the platform MBean server creates an access path to WebLogic Server MBeans that is not secured through the WebLogic Server security framework."
JVM Platform MBean Server
Someone on the internet says the following: "Note that the Platform MBean Server is always present and is created by the JVM during startup." But I am not sure about the meaning of "always present" here. It seems to me that by default, JVM will not create the MBean server during the startup. I did a simple test by using JDK1.7. I created a simple class Temp1.java that just calls the Thread.sleep method so it will not stop immediately after being executed. Then if I simply run the class using the command "java Temp1", jconsole will fail to connect to it. Only if I use the command "java -Dcom.sun.management.jmxremote Temp1", will the jconsole be able to connect to it. After doing more searching, I found the following from java7 doc:
In the Java SE 6 platform, it is no longer necessary to set this system property. Any application that is started on the Java SE 6 platform will support the Attach API, and so will automatically be made available for local monitoring and management when needed.
In the above, the "system property" refers to "com.sun.management.jmxremote". For remote access, the document is the following:
To enable monitoring and management from remote systems, you must set the following system property when you start the Java VM.
com.sun.management.jmxremote.port=portNum
In the property above, portNum is the port number through which you want to enable JMX RMI connections. Be sure to specify an unused port number. In addition to publishing an RMI connector for local access, setting this property publishes an additional RMI connector in a private read-only registry at the specified port using a well known name, "jmxrmi".
The Attach API mentioned above is worth studying. Ref[4] has some example code about the Attach API. The main ideas are the following.
- The Attach API is provided in the tools.jar file from JDK. The main class is VirtualMachine.
- The VirtualMachine class represents a specific Java virtual machine(JVM) instance. You connect to a JVM by providing the VirtualMachine class with the process id, and then you load a management agent to do your customized behavior.
- After you load the agent with loadAgent, you should call the detach method.
- A JMX agent exists in the managment-agent.jar file that comes with the JDK found in the same directory as tools.jar. The sample code is the following:
VirtualMachine vm = VirtualMachine.attach(args[0]); String connectorAddr = vm.getAgentProperties().getProperty( "com.sun.management.jmxremote.localConnectorAddress"); if (connectorAddr == null) { String agent = vm.getSystemProperties().getProperty( "java.home")+File.separator+"lib"+File.separator+ "management-agent.jar"; vm.loadAgent(agent); connectorAddr = vm.getAgentProperties().getProperty( "com.sun.management.jmxremote.localConnectorAddress"); } JMXServiceURL serviceURL = new JMXServiceURL(connectorAddr); JMXConnector connector = JMXConnectorFactory.connect(serviceURL); MBeanServerConnection mbsc = connector.getMBeanServerConnection(); ObjectName objName = new ObjectName( ManagementFactory.THREAD_MXBEAN_NAME); SetIn the above, args[0] should be the process ID of the target JVM.mbeans = mbsc.queryNames(objName, null); for (ObjectName name: mbeans) { ThreadMXBean threadBean; threadBean = ManagementFactory.newPlatformMXBeanProxy( mbsc, name.toString(), ThreadMXBean.class); long threadIds[] = threadBean.getAllThreadIds(); for (long threadId: threadIds) { ThreadInfo threadInfo = threadBean.getThreadInfo(threadId); System.out.println (threadInfo.getThreadName() + " / " + threadInfo.getThreadState()); }
It looks like that lots of operating system level functions are needed in the back of the scene for the code to work. You run the code containing the VirtualMachine class in your JVM to connect to the target JVM whose process ID is known to you. At the level of operating system, this is the communication between two processes. There is no port number involved in the above code. There is no TCP/IP in the above code.
One interesting consequence of the Attach API is this. Say that you run a program such as jconsole to try to connect to a JVM to get the JMX MBean server. Suppose that the connection works. Can you conclude from this that the MBean server has been started when the JVM is started? The answer is "NO". This is because that the MBean sever may have been started by your client program by using the Attach API!
References
- http://java.dzone.com/articles/taming-jmx-weblogic-server
- http://docs.spring.io/spring/docs/2.0.x/reference/jmx.html
- http://docs.oracle.com/cd/E11035_01/wls100/jmx/understandWLS.html#MBeanServers
- http://www.javaworld.com/article/2071330/the-attach-api.html
