Blog's Fourth Anniversay

Today is the fourth year anniversary of this blog. It is cloudy. The current temperature is 76F. This summer in Pittsburgh has been rainy and not very hot. But yesterday was quite humid with high temperature. I planted one seedling for small tomato in the garden the first time this year. I have harvested about 60 so far.

I just came back from seeing the dentist. I was recommended to have a tooth guard. Recently I have spent quite some time studying algorithms and JMS.

Spring JMS

The General Spring Bean Life Cycle in a Web Application

The java web application provides a plugin point in the web.xml for the user to supply his own context. It is the <listener> tag where a user can put his own class that implements the interface javax.servlet.ServletContextListener. The implementation in Spring is the ContextLoaderListener class. And the user can pass additional parameter values to his listener using the <context-param> tag. The following is an example:

<context-param>
  <param-name>contextConfigLocation</param-name>
  <param-value>
            classpath:com/xyz/dao/applicationContext-dao.xml    
            classpath:com/xyz/jms/applicationContext-jms.xml    
  </param-value>
 </context-param>

 <listener>
  <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
 </listener>

Bean Creation

The ContextLoaderListener uses org.springframework.web.context.ContexLoader by default to perform the actual initialization work for the root application context.

ContextLoaderListener implements the following API method from the interface ServletContextListener. And this method will create the Spring beans when the web application is started.

public void contextInitialized(ServletContextEvent event) {
  this.contextLoader = createContextLoader();
  this.contextLoader.initWebApplicationContext(event.getServletContext());
 }

The following is the code of the method initWebApplicationContext of the class ContextLoader:

public WebApplicationContext initWebApplicationContext(ServletContext servletContext)
   throws IllegalStateException, BeansException {

  if (servletContext.getAttribute(WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE) != null) {
   throw new IllegalStateException(
     "Cannot initialize context because there is already a root application context present - " +
     "check whether you have multiple ContextLoader* definitions in your web.xml!");
  }
                 ......
   // Determine parent for root web application context, if any.
   ApplicationContext parent = loadParentContext(servletContext);

   // Store context in local instance variable, to guarantee that
   // it is available on ServletContext shutdown.
   this.context = createWebApplicationContext(servletContext, parent);
   servletContext.setAttribute(WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE, this.context);
   currentContextPerThread.put(Thread.currentThread().getContextClassLoader(), this.context);
                ......
   return this.context;
  
 }

You can see that Spring creates its own context and stores it in the standard servletContext object. This way it can be retrieved in other code from the servletContext. The code first has a call to loadParentContext. This is useful if the web module WAR is part of an EAR application. The following is the javadoc from Spring for this method:

The main reason to load a parent context here is to allow multiple root web application contexts to all be children of a shared EAR context, or alternately to also share the same parent context that is visible to EJBs. For pure web applications, there is usually no need to worry about having a parent context to the root web application context.

The following code shows how to get the Spring bean from the bean ID using the context:

import org.springframework.web.context.WebApplicationContext;
import org.springframework.web.context.support.WebApplicationContextUtils;

public static Object getBean(String beanId) {
  
  FacesContext facesContext = FacesContext.getCurrentInstance();
  ServletContext servletContext = (ServletContext) facesContext
    .getExternalContext().getContext();
  WebApplicationContext webApplicationContext = WebApplicationContextUtils
    .getWebApplicationContext(servletContext);
  return webApplicationContext.getBean(beanId);
 }

The above method is used in JSF(Java Server Faces), hence the FacesContext. If you are not in JSF, you can also do similar thing as long as you have the servletConext object. Basically it will use the same attribute WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE to get the context from the servletContext. The actual code of the method getWebApplicationContext is below:

public static WebApplicationContext getWebApplicationContext(ServletContext sc) {
  return getWebApplicationContext(sc, WebApplicationContext.ROOT_WEB_APPLICATION_CONTEXT_ATTRIBUTE);
 }

The Spring beans are defined in xml files. How are these bean names because the actual java class objects? Spring uses the class loaders that can be got by calling Thread.currentThread().getContextClassLoader() or class.getClassLoader() to load the classes.

The Spring beans are usually singletons. For different HTTP requests from the web clients, different threads are used to serve the requests. But the beans used in these threads are the same. In Weblogic, you may see sometimes that the threads to serve the clients are also same. I think that is because Weblogic maintains a thread pool. So the same thread may be used for two different requests.

Bean Termination

ContextLoaderListener implements the following API method from the interface ServletContextListener. And this method will destroy the Spring beans when the web application is shutdown.

public void contextDestroyed(ServletContextEvent event) {
  if (this.contextLoader != null) {
   this.contextLoader.closeWebApplicationContext(event.getServletContext());
  }
 }

The following is the actual stack when I redeploy a web application to the weblogic server. You can see the call stack to create beans.

   DefaultListableBeanFactory(DefaultSingletonBeanRegistry).getSingleton(String, ObjectFactory) line: 222 
   DefaultListableBeanFactory(AbstractBeanFactory).doGetBean(String, Class, Object[], boolean) line: 261 
   DefaultListableBeanFactory(AbstractBeanFactory).getBean(String, Class, Object[]) line: 185 
   DefaultListableBeanFactory(AbstractBeanFactory).getBean(String) line: 164 
   DefaultListableBeanFactory.preInstantiateSingletons() line: 429 
   XmlWebApplicationContext(AbstractApplicationContext).finishBeanFactoryInitialization(ConfigurableListableBeanFactory) line: 728 
   XmlWebApplicationContext(AbstractApplicationContext).refresh() line: 380 
   ContextLoader.createWebApplicationContext(ServletContext, ApplicationContext) line: 255 
   ContextLoader.initWebApplicationContext(ServletContext) line: 199 
   ContextLoaderListener.contextInitialized(ServletContextEvent) line: 45 
   EventsManager$FireContextListenerAction.run() line: 481 
   AuthenticatedSubject.doAs(AbstractSubject, PrivilegedAction) line: 321 
   SecurityManager.runAs(AuthenticatedSubject, AuthenticatedSubject, PrivilegedAction) line: 121 
   EventsManager.notifyContextCreatedEvent() line: 181 
   WebAppServletContext.preloadResources() line: 1801 
   WebAppServletContext.start() line: 3045 
   WebAppModule.startContexts() line: 1397 
   WebAppModule.start() line: 460 

JMS Sender

Send XML message using JAXB marshalling with XML schema validation

The following is the sample code to use the Spring JmsTemplate class to send the XML message. The XML message is created by marshalling a java Foo ojbect. The class Foo is a generated class from JAXB using the XSD file "your.xsd". The marshaller loads the XSD file so it will validate the XML content. If the java foo object does not create a valid message, the method "marshaller.marshal(xmlMsg, sw)" will throw exception. For example, if a field is required in the XSD file your.xsd, and the foo ojbect has null value for that field, then a SAXParseException will be thrown. The file schemas/your.xsd must be put on the classpath.

Also note that in this example, the JMS message type is BytesMessage. It also creates properties in the JMS message.

import org.springframework.jms.core.JmsTemplate;
import org.springframework.jms.core.MessageCreator;

public void sendMessage(final String message, final Map properties) {
  jmsTemplate.send(new MessageCreator() {
   @Override
   public Message createMessage(Session session) throws JMSException {
    BytesMessage bytesMsg = session.createBytesMessage();
    bytesMsg.writeBytes(message.getBytes());
    if (properties != null && !properties.isEmpty()) {
     Iterator it = properties.entrySet().iterator();
     while (it.hasNext()) {
      Map.Entry entry = (Map.Entry) it.next();
      String key = (String) entry.getKey();
      String val = (String) entry.getValue();
      bytesMsg.setStringProperty(key, val);
     }
    }
    return bytesMsg;
   }
  });
 }

 public void sendMessage(Foo fooObj, final Map properties)
   throws JAXBException {
  try {
   String message = marshall(fooObj);
   this.sendMessage(message, properties);
  } catch (SAXException e) {
   throw new JAXBException(e);
  }
 }

 private String marshall(Foo fooObj) throws JAXBException,
   SAXException {
  JAXBElement xmlMsg = new com.example.schema.ObjectFactory()
    .createXYZ(fooObj);
  JAXBContext jaxbContext = JAXBContext.newInstance(Foo.class);
  Marshaller marshaller = jaxbContext.createMarshaller();
  SchemaFactory schemaFactory = SchemaFactory
    .newInstance(XMLConstants.W3C_XML_SCHEMA_NS_URI);
  URL xsdUrl = getClass().getClassLoader().getResource(
    "schemas/your.xsd");
  Schema schema = schemaFactory.newSchema(xsdUrl);
  marshaller.setSchema(schema);
  StringWriter sw = new StringWriter();
  marshaller.marshal(xmlMsg, sw);
  return sw.toString();
 }

The Life Cycle of JMS Connection and Session

The following is a case study. The version Spring 2.5.6 is used. In the actual application code,the operations are put into a JTA transaction. The analysis below is to show how the JMS connection and session are created and closed.

1. Spring uses JmsTemplate to send messages
2. JmsTemplate uses the following method to actually send the message:

       public Object execute(SessionCallback action, boolean startConnection) throws JmsException {
        ...
         
        Session sessionToUse = ConnectionFactoryUtils.doGetTransactionalSession(
            getConnectionFactory(), this.transactionalResourceFactory, startConnection);
        ...
        return action.doInJms(sessionToUse);
           ...  
   
        finally {
          JmsUtils.closeSession(sessionToClose);
          ConnectionFactoryUtils.releaseConnection(conToClose, getConnectionFactory(), startConnection);
        }
      }
    

   Notes:
   (1) The "finally" block closes sessionToClose and conToClose. These are not the same as sessionToUse and its connection. When sessionToUse is not null, sessionToClose and conToClose are not created, and the "finally" block will do nothing when they are null.
   (2) I did not find a call to connection.start(). I did a separate test and it showed that the message can still be sent if start() is not invoked. But for a JMS consumer, this has to be called. Otherwise, no message will be received.
3. The call ConnectionFactoryUtils.doGetTransactionalSession(...) creates the connection and the session. And the connection is saved into a new JmsResourceHolder() which is bound to TransactionSynchronizationManager in the following code in that method:

      TransactionSynchronizationManager.bindResource(connectionFactory, resourceHolderToUse);
    

4. At the end of the operations, TransactionSynchronizationManager calls the following method:

         public static Object unbindResource(Object key) 
         

   Eventually the following code of class JmsResourceHolder is called:

       public void closeAll() {
         for (Iterator it = this.sessions.iterator(); it.hasNext();) {
          try {
           ((Session) it.next()).close();
          }
          catch (Throwable ex) {
           logger.debug("Could not close synchronized JMS Session after transaction", ex);
          }
         }
         for (Iterator it = this.connections.iterator(); it.hasNext();) {
          Connection con = (Connection) it.next();
          ConnectionFactoryUtils.releaseConnection(con, this.connectionFactory, true);
         }
         this.connections.clear();
         this.sessions.clear();
         this.sessionsPerConnection.clear();
    }
       

Question: What if the connection returned from the ConnectionFactory is a pooled connection? Obviously it should not be closed at the end. How does Spring know it is pooled and not close it?

One implementation I saw that solve the question above is the following. The code ConnectionFactoryUtils.releaseConnection(con, this.connectionFactory, true); will call con.close();. But the trick is in the method close(). The connection is a customer-implemented class. Its "close" method actually does not close it in all circumstances. The connection class itself has a reference to the connection factory. It will first check if connection pooling is enabled. If yes, it will just tell the factory to update the status for this connection, and that is it. Only if the pooling is not enabled, will the connection actually close itself.

Spring also has its own JMS connection pool implementation. One of them is the SingleConnectionFactory. It uses the similar way as the one mentioned above to avoid the connection being closed. It will return a connection proxy that will essentially do nothing in the "close()" method. The following is the actual code:

/**
  * Wrap the given Connection with a proxy that delegates every method call to it
  * but suppresses close calls. This is useful for allowing application code to
  * handle a special framework Connection just like an ordinary Connection from a
  * JMS ConnectionFactory.
  * @param target the original Connection to wrap
  * @return the wrapped Connection
  */
 protected Connection getSharedConnectionProxy(Connection target) {
  List classes = new ArrayList(3);
  classes.add(Connection.class);
  if (target instanceof QueueConnection) {
   classes.add(QueueConnection.class);
  }
  if (target instanceof TopicConnection) {
   classes.add(TopicConnection.class);
  }
  return (Connection) Proxy.newProxyInstance(
    Connection.class.getClassLoader(),
    (Class[]) classes.toArray(new Class[classes.size()]),
    new SharedConnectionInvocationHandler(target));
 }

The following javadoc of the class org.springframework.jms.connection.SingleConnectionFactory is also quite helfpful.

A JMS ConnectionFactory adapter that returns the same Connection from all 
createConnection() calls, and ignores calls to Connection.close(). According to the
 JMS Connection model, this is perfectly thread-safe (in contrast to e.g. JDBC). The 
shared Connection can be automatically recovered in case of an Exception. 

You can either pass in a specific JMS Connection directly or let this factory lazily 
create a Connection via a given target ConnectionFactory. This factory generally works
 with JMS 1.1 as well as the JMS 1.0.2 API. 

...

Useful for testing and standalone environments in order to keep using the same 
Connection for multiple JmsTemplate calls, without having a pooling ConnectionFactory
 underneath. This may span any number of transactions, even concurrently executing 
transactions. 

Note that Spring's message listener containers support the use of a shared Connection
 within each listener container instance. Using SingleConnectionFactory in combination
 only really makes sense for sharing a single JMS Connection across multiple listener
 containers. 

So the answer to the original question may be the following. Spring actually does not detect if a connection returned from a connection factory is pooled or not. It just calls its "close" method. It lets the connection class itself to decide whether or not to close the actual connection.

JMS Consumer

The Life Cycle of JMS Connection and Session

Spring uses DefaultMessageListenerContainer to receiver messages. An example is as follows:

<bean id="jmsContainer"
  class="org.springframework.jms.listener.DefaultMessageListenerContainer">
  <property name="connectionFactory" ref="jmsDestConnectionFactory" />
  <property name="destination" ref="myDestination" />
  <property name="sessionAcknowledgeModeName" value="AUTO_ACKNOWLEDGE" />
  <property name="messageListener" ref="messageListener" />
  <property name="sessionTransacted" value="true" />
  <property name="concurrentConsumers" value="1"></property>
 </bean>

The main steps are the following.

1. When the bean is created, the JMS connection and session are created. The connection is also started.
2. The conneciton will be reused by the application. When the application is shutdown, the connection will be stopped and then closed.

The following are the main steps that the JMS connection is closed.

1. org.springframework.web.context.ContextLoader calls

       public void closeWebApplicationContext(ServletContext servletContext) {...}
   

2. The method above will call

    protected void doClose() {
         ...
          Map lifecycleBeans = getLifecycleBeans();
          for (Iterator it = new LinkedHashSet(lifecycleBeans.keySet()).iterator(); it.hasNext();) {
           String beanName = (String) it.next();
           doStop(lifecycleBeans, beanName);
          }
          // Destroy all cached singletons in the context's BeanFactory.
          destroyBeans();
          // Close the state of this context itself.
          closeBeanFactory();
          onClose();
          synchronized (this.activeMonitor) {
           this.active = false;
          }
         }
    }
   

   In the above code, the "doStop()" method will stop the connection, and the "destoryBeans()" will close the connection

What Happens Internally in JMS Connection and Session?

So how are the messages actually sent over the network? We have to use open source code to get some idea. I looked at ActiveMQ, which is a popular open source JMS implementation. The basic steps to send a message are as follows:

1. MessageProducer invokes the "send(...)" method.
2. The MessageProducer implementation actually has a reference to the session. So its "send" method will call "session.send(...)"
3. The session implementation actually has a reference to the connection. So it calls "connection.synSendPacket(...)" or "connection.aSyncSendPacket(...)".
4. The connection object will delegate the task to some transport layer object to actually send the message. And eventually I saw in a place that it uses ObjectOutputStream to send out the message.

Other observations on ActiveMQ.

1. A connection holds references to all the sessions that it has created.
2. A session holds references to all the consumers that it has created.
3. When a connection is started, it calls the "start()" method of all its sessions.
4. When a session is started, it calls the "start()" method of all its consumers. Also it calls "executor.start()". The executor is of class ActiveMQSessionExecutor. Its "start()" method has the following main code line:

      messageQueue.start();
   

   The messageQueue here is of class MessageDispatchChannel.
5. The "start()" method of the consumer has the following main code lines:

         unconsumedMessages.start();
         session.executor.wakeup();
   

   The unconsumerdMessages is an instance of MessageDispatchChannel.
6. The MessageDispatchChannel interface has the "enqueue" and "dequeue" methods. One implementation of this interface has a list of MessageDispatch, which has an instance variable of Message. So this list can be used to store and send messages. But there must be other ways such as using files or a database to store the messages. I am not clear here.

Transaction for Spring JMS consumer

The general JMS API specifies that when you create a session, you specify whether it is transacted. The first argument to the createSession method is a boolean value. A value of true means that the session is transacted; a value of false means that it is not transacted. The second argument to this method is the acknowledge mode, which is relevant only to nontransacted sessions. If the session is transacted, the second argument is ignored.

The commit and the rollback methods for local transactions are associated with the session.

In an Enterprise JavaBeans component, you cannot use the Session.commit and Session.rollback methods. Instead, you use distributed transactions.

Now for Spring class org.springframework.jms.listener.DefaultMessageListenerContainer, it has the properties transactionManager, sessionAcknowledgeModeName and sessionTransacted. The logic about how these properties are used are the following:

1. If the property "transactionManager" is set, then that transaction manager will be used. The property "sessionTransacted" will be ignored.
2. Else if "sessionTransacted" is set to "true", local transaction will be used. And the property "sessionAcknowledgeModeName" will be ignored. Programs need to use session.commit or session.rollback to manage the transaction.
3. Else there is no transaction. The property "sessionAcknowledgeModeName" comes into effect.

The javadoc[1] of Spring recommends the following:

1. The general recommendation is to set "sessionTransacted" to "true", typically in combination with local database transactions triggered by the listener implementation, through Spring's standard transaction facilities. This will work nicely in Tomcat or in a standalone environment, often combined with custom duplicate message detection (if it is unacceptable to ever process the same message twice).
2. Alternatively, specify a JtaTransactionManager as "transactionManager" for a fully XA-aware JMS provider - typically when running on a J2EE server, but also for other environments with a JTA transaction manager present. This will give full "exactly-once" guarantees without custom duplicate message checks, at the price of additional runtime processing overhead.

   Note that it is also possible to specify a JmsTransactionManager as external "transactionManager", providing fully synchronized Spring transactions based on local JMS transactions. The effect is similar to "sessionTransacted" set to "true", the difference being that this external transaction management will also affect independent JMS access code within the service layer (e.g. based on JmsTemplate or TransactionAwareConnectionFactoryProxy), not just direct JMS Session usage in a SessionAwareMessageListener.

Note from [1] about how the exception is handled for the class AbstractMessageListenerContainer, which is an upper class of DefaultMessageListenerContainer: "NOTE: The default behavior of this message listener container is to never propagate an exception thrown by a message listener up to the JMS provider. Instead, it will log any such exception at the error level. This means that from the perspective of the attendant JMS provider no such listener will ever fail."

The Duplicate Messages Issue

Again from [1], I think the following is true for the DefaultMessageListenerContainer. Note that cases #4, #5 and #6 are only effective when no transaction manager or local transaction is used. If a transactionManager is set or the sessionTransacted is set to "true", the AcknowledgeMode will be ignored.

1. If a JtaTransactionManager as "transactionManager" for a fully XA-aware JMS provider, it will give full "exactly-once" guarantees without custom duplicate message checks
2. To specify a JmsTransactionManager as external "transactionManager". The javadoc does not explicitly say if there will be duplicate messages. But it says this case is similar to "sessionTransacted" set to "true", in which case the duplicate messaegs are possible.
3. To set "sessionTransacted" to "true". In this case, the duplicate messages are possible because the javadoc there says this case is "often combined with custom duplicate message detection"
4. "sessionAcknowledgeMode" set to "AUTO_ACKNOWLEDGE" (default): Automatic message acknowledgment before listener execution; no redelivery in case of exception thrown.
5. "sessionAcknowledgeMode" set to "CLIENT_ACKNOWLEDGE": Automatic message acknowledgment after successful listener execution; no redelivery in case of exception thrown.
6. "sessionAcknowledgeMode" set to "DUPS_OK_ACKNOWLEDGE": Lazy message acknowledgment during or after listener execution; potential redelivery in case of exception thrown.

Also note that the javadoc has the following regarding the above acknowledge modes: "The exact behavior might vary according to the concrete listener container and JMS provider used."

Appendix

To see more about the steps of how a Spring bean is created, the following complete stack trace is helpful. The stack is obtained by putting a break point in the code and redeploying the web application to the weblogic server that is started in debug mode.

Oracle WebLogic Server 11gR1 PatchSet 1 at localhost [Oracle WebLogic Launch Configuration] 
 Java HotSpot(TM) Client VM[localhost:8453] 
  Daemon Thread [DynamicListenThread[Default]] (Running) 
  Daemon Thread [DynamicListenThread[Default[2]]] (Running) 
  Daemon Thread [DynamicListenThread[Default[3]]] (Running) 
  Daemon Thread [DynamicListenThread[Default[1]]] (Running) 
  Daemon Thread [[STANDBY] ExecuteThread: '4' for queue: 'weblogic.kernel.Default (self-tuning)'] (Running) 
  Daemon Thread [weblogic.GCMonitor] (Running) 
  Daemon Thread [TIBCO EMS TCPLink Reader (Server-47729)] (Running) 
  Daemon Thread [TIBCO EMS Connections Pinger] (Running) 
  Daemon Thread [Timer-2] (Suspended (exception NoClassDefFoundError)) 
   TimerThread.run() line: 468 
  Daemon Thread [FooJMSStarvedConsumerTimer] (Running) 
  Daemon Thread [[ACTIVE] ExecuteThread: '3' for queue: 'weblogic.kernel.Default (self-tuning)'] (Running) 
  Daemon Thread [Monitor Runner] (Running) 
  Daemon Thread [Session Monitor] (Running) 
  Daemon Thread [Session Monitor] (Running) 
  Daemon Thread [[STANDBY] ExecuteThread: '2' for queue: 'weblogic.kernel.Default (self-tuning)'] (Suspended (breakpoint at line 37 in SampleProducer)) 
   SampleProducer.setJmsTemplate(JmsTemplate) line: 37 
   NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method] 
   NativeMethodAccessorImpl.invoke(Object, Object[]) line: 39 
   DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 25 
   Method.invoke(Object, Object...) line: 597 
   BeanWrapperImpl.setPropertyValue(BeanWrapperImpl$PropertyTokenHolder, PropertyValue) line: 840 
   BeanWrapperImpl.setPropertyValue(PropertyValue) line: 651 
   BeanWrapperImpl(AbstractPropertyAccessor).setPropertyValues(PropertyValues, boolean, boolean) line: 78 
   BeanWrapperImpl(AbstractPropertyAccessor).setPropertyValues(PropertyValues) line: 59 
   DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).applyPropertyValues(String, BeanDefinition, BeanWrapper, PropertyValues) line: 1276 
   DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).populateBean(String, AbstractBeanDefinition, BeanWrapper) line: 1010 
   DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).doCreateBean(String, RootBeanDefinition, Object[]) line: 472 
   AbstractAutowireCapableBeanFactory$1.run() line: 409 
   AccessController.doPrivileged(PrivilegedAction, AccessControlContext) line: not available [native method] 
   DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).createBean(String, RootBeanDefinition, Object[]) line: 380 
   AbstractBeanFactory$1.getObject() line: 264 
   DefaultListableBeanFactory(DefaultSingletonBeanRegistry).getSingleton(String, ObjectFactory) line: 222 
   DefaultListableBeanFactory(AbstractBeanFactory).doGetBean(String, Class, Object[], boolean) line: 261 
   DefaultListableBeanFactory(AbstractBeanFactory).getBean(String, Class, Object[]) line: 185 
   DefaultListableBeanFactory(AbstractBeanFactory).getBean(String) line: 164 
   DefaultListableBeanFactory.findAutowireCandidates(String, Class, DependencyDescriptor) line: 671 
   DefaultListableBeanFactory.resolveDependency(DependencyDescriptor, String, Set, TypeConverter) line: 610 
   AutowiredAnnotationBeanPostProcessor$AutowiredMethodElement.inject(Object, String, PropertyValues) line: 499 
   InjectionMetadata.injectMethods(Object, String, PropertyValues) line: 117 
   AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(PropertyValues, PropertyDescriptor[], Object, String) line: 253 
   DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).populateBean(String, AbstractBeanDefinition, BeanWrapper) line: 998 
   DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).doCreateBean(String, RootBeanDefinition, Object[]) line: 472 
   AbstractAutowireCapableBeanFactory$1.run() line: 409 
   AccessController.doPrivileged(PrivilegedAction, AccessControlContext) line: not available [native method] 
   DefaultListableBeanFactory(AbstractAutowireCapableBeanFactory).createBean(String, RootBeanDefinition, Object[]) line: 380 
   AbstractBeanFactory$1.getObject() line: 264 
   DefaultListableBeanFactory(DefaultSingletonBeanRegistry).getSingleton(String, ObjectFactory) line: 222 
   DefaultListableBeanFactory(AbstractBeanFactory).doGetBean(String, Class, Object[], boolean) line: 261 
   DefaultListableBeanFactory(AbstractBeanFactory).getBean(String, Class, Object[]) line: 185 
   DefaultListableBeanFactory(AbstractBeanFactory).getBean(String) line: 164 
   DefaultListableBeanFactory.preInstantiateSingletons() line: 429 
   XmlWebApplicationContext(AbstractApplicationContext).finishBeanFactoryInitialization(ConfigurableListableBeanFactory) line: 728 
   XmlWebApplicationContext(AbstractApplicationContext).refresh() line: 380 
   ContextLoader.createWebApplicationContext(ServletContext, ApplicationContext) line: 255 
   ContextLoader.initWebApplicationContext(ServletContext) line: 199 
   ContextLoaderListener.contextInitialized(ServletContextEvent) line: 45 
   EventsManager$FireContextListenerAction.run() line: 481 
   AuthenticatedSubject.doAs(AbstractSubject, PrivilegedAction) line: 321 
   SecurityManager.runAs(AuthenticatedSubject, AuthenticatedSubject, PrivilegedAction) line: 121 
   EventsManager.notifyContextCreatedEvent() line: 181 
   WebAppServletContext.preloadResources() line: 1801 
   WebAppServletContext.start() line: 3045 
   WebAppModule.startContexts() line: 1397 
   WebAppModule.start() line: 460 
   ModuleStateDriver$3.next(Object) line: 425 
   StateMachineDriver.nextState(StateChange, Object[]) line: 83 
   ModuleStateDriver.start(Module[]) line: 119 
   ScopedModuleDriver.start() line: 200 
   ModuleListenerInvoker.start() line: 247 
   ModuleStateDriver$3.next(Object) line: 425 
   StateMachineDriver.nextState(StateChange, Object[]) line: 83 
   ModuleStateDriver.start(Module[]) line: 119 
   StartModulesFlow.activate() line: 27 
   BaseDeployment$2.next(Object) line: 1267 
   StateMachineDriver.nextState(StateChange, Object[]) line: 83 
   WarDeployment(BaseDeployment).activate(DeploymentContext) line: 409 
   WarDeployment(SingleModuleDeployment).activate(DeploymentContext) line: 39 
   DeploymentStateChecker.activate(DeploymentContext) line: 161 
   AppContainerInvoker.activate(DeploymentContext) line: 79 
   RedeployOperation(AbstractOperation).activate(Deployment) line: 569 
   RedeployOperation(ActivateOperation).activateDeployment() line: 150 
   RedeployOperation(ActivateOperation).doCommit() line: 116 
   RedeployOperation(AbstractOperation).commit() line: 323 
   DeploymentManager.handleDeploymentCommit(Deployment, AbstractOperation) line: 844 
   DeploymentManager.activateDeploymentList(ArrayList, DeploymentManager$DeploymentRequestInfo) line: 1253 
   DeploymentManager.handleCommit(DeploymentContext) line: 440 
   DeploymentServiceDispatcher.commit(DeploymentContext) line: 163 
   DeploymentReceiverCallbackDeliverer.doCommitCallback(DeploymentContext) line: 181 
   DeploymentReceiverCallbackDeliverer.access$100(DeploymentReceiverCallbackDeliverer, DeploymentContext) line: 12 
   DeploymentReceiverCallbackDeliverer$2.run() line: 67 
   SelfTuningWorkManagerImpl$WorkAdapterImpl.run() line: 516 
   ExecuteThread.execute(Runnable) line: 201 
   ExecuteThread.run() line: 173 
  Daemon Thread [AS400 Read Daemon [system:pitt2;job:114314/QUSER/QZDASOINIT]] (Running) 
  Daemon Thread [AS400 Read Daemon [system:tst1;job:113896/QUSER/QZDASOINIT]] (Running) 
  Daemon Thread [Thread-11] (Running) 
  Daemon Thread [DoSManager] (Running) 
  Daemon Thread [VDE Transaction Processor Thread] (Running) 
  Daemon Thread [ExecuteThread: '3' for queue: 'weblogic.socket.Muxer'] (Running) 
  Daemon Thread [ExecuteThread: '2' for queue: 'weblogic.socket.Muxer'] (Running) 
  Daemon Thread [ExecuteThread: '1' for queue: 'weblogic.socket.Muxer'] (Running) 
  Daemon Thread [ExecuteThread: '0' for queue: 'weblogic.socket.Muxer'] (Running) 
  Daemon Thread [Thread-7] (Running) 
  Daemon Thread [[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Default (self-tuning)'] (Running) 
  Daemon Thread [weblogic.timers.TimerThread] (Running) 
  Daemon Thread [weblogic.time.TimeEventGenerator] (Running) 
  Daemon Thread [[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'] (Running) 
  Daemon Thread [Timer-1] (Running) 
  Daemon Thread [Timer-0] (Running) 
  Thread [main] (Running) 
  Daemon Thread [FooJMSStarvedConsumerTimer] (Running) 
 Weblogic Server 
new hap-ws LDEV [Maven Build] 
 C:\bea1032\jdk160_14_R27.6.5-32\bin\javaw.exe (Sep 10, 2013 4:59:32 PM) 

References

1. http://docs.spring.io/spring/docs/2.5.6/api/org/springframework/jms/listener/AbstractMessageListenerContainer.html

Partition, File System, Directory, etc

Partition and File System

Found the following link that has good explanations on partition, file system, etc. Though it is about UNIX, the concepts on LINUX should be similar.
http://www.freebsd.org/doc/handbook/disk-organization.html

The main points are the following.

1. Directories and files are stored in a file system. Each file system contains exactly one directory at the very top level, called the root directory for that file system.
2. One file system is designated the root file system or /. Every other file system is mounted under the root file system.
3. No matter how many disks are on the FreeBSD system, every directory appears to be part of the same disk.
4. File systems can be mounted on top of one another.
5. File systems are contained in partitions.
6. Each partition can contain only one file system.
7. File systems are a fixed size.

You can use the command "df" to get the information of the file systems on a server. The following is an example on a LINUX server.

$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda3     ext3     3882516   2032072   1650036  56% /
/dev/mapper/vg00-optvol
              ext3     8125880   2049796   5656656  27% /opt
/dev/mapper/vg00-homevol
              ext3     1015704     34176    929100   4% /home
/dev/mapper/vg00-varvol
              ext3     8125880   1960524   5745928  26% /var
/dev/sda1     ext3      194410     17867    166506  10% /boot
tmpfs        tmpfs     1956764         0   1956764   0% /dev/shm
tmpfs        tmpfs     2097152      3716   2093436   1% /tmp
/dev/mapper/vg01-lvol1
              ext3     1032088     34936    944724   4% /opt/abc
/dev/mapper/vg01-lvol2
              ext3     1032088     34100    945560   4% /var/abc
/dev/mapper/vg00-UsrLclHmVol
              ext3     5160576   1566824   3331608  32% /usr/local/home
/dev/mapper/vg01-AppOptVol
              ext3    10321208    399984   9396936   5% /opt/abc/hzm
/dev/mapper/vg01-AppVarVol
              ext3    10321208   1388868   8408052  15% /var/abc/hzm
pghplnxl01:/export/kickstart
               nfs   131272224 120520768   4086592  97% /mnt/install

In the above example, the Type values are mostly "ext3". This does not have to be true. The following example is from Reference[3]

$ df -T
Filesystem    Type   1K-blocks      Used Available Use% Mounted on
/dev/sda1     ext4   132239776   6210892 119311496   5% /
tmpfs        tmpfs     4021876         0   4021876   0% /dev/shm
/dev/sdb2     ext2    30969600    117740  29278696   1% /home/oracle
/dev/sdc1     ext2   576310180     71232 546964104   1% /home/data

Swap Space

1. Linux has two forms of swap space: the swap partition and the swap file.The swap partition is an independent section of the hard disk used solely for swapping; no other files can reside there. The swap file is a special file in the filesystem that resides amongst your system and data files.
2. To see what swap space you have, use the command "swapon -s".

The following is an example to find the swap space. It is performed on the same machine as in the above section.

$ /sbin/swapon -s
Filename                                Type            Size    Used    Priority
/dev/sda2                               partition       457824  457820  -1

The output above shows a partition. It is not a swap file.
Note that the "df -T" command will not show the partition information.

Find Sizes of the Files

The command "du" can be used to find the estimated file space usage. The following command will write counts for all files and sub-directories under the current directory.

du -a

References

1. http://www.ibm.com/developerworks/linux/library/l-lpic1-v3-104-1/index.html
2. http://www.linux.com/news/software/applications/8208-all-about-linux-swap-space
3. http://www.thegeekstuff.com/2012/05/df-examples

Nest for-loop n-times, recursive calls, and beyond

It is easy to write the code to nest the for-loop n-times if you know the value of n. For example, if you know n=3, then the code is as follows:

   for(int i=0; i<length; i++){
       for(int j=0; j<length; j++){
          for(int k=0; k<length; k++){
          }
       }
   }

But what if the value of n is just a variable? How to write code for this? It can be solved using recursive function. The basic code is as follows:

 public void func(int count, int n){
    if( count == n){
       return;
    }
    for(int i=0; i<length; i++){
        func(count+1,n);
     }
 }
 

And you just need to call func(0,n); The basic idea here is to pass the count number as an argument to the next recursive call. And the next recursive call can determine if to finish or continue based on that value. Here we only pass the count number to the subsequent call. We can also pass other values in the arguments to the next recursive call and solve some quite complex problems! If we make some twists in the actual body of the function, this code can be even more powerful! We show several algorithms here.

N-Queen Problem

Problem: Position n queens on a chessboard so that no two are in the same row, column, or diagonal.
Inputs: positive integer n.
Outputs: All possible ways n queens can be placed on an n*n chessboard so that no two queens threaten each other.
Solution: We use col[i] to denote the column position of the queen in i-th row. We will pass col[] as an argument to the next recursive call so it can be used to check if the positions are valid.

 public void func(int count, int[] col){
    if ( count == col.length){
       printOutResult(col);
       return;
    }
    for(int i=0; i<col.length; i++){
       col[count] = i;
       if ( goodSoFar(col, count)){
          func(count+1, col);
       }
    }
 }
 
 private void printOutResult(int[] col){
    for(int i=0; i<col.length; i++){
      System.out.println(" " + col[i]);
    }
 }
 
 private boolean goodSoFar(int[] col, int count){
   for(int i=0; i<count; i++){
      if ( col[i] == col[count]){ // same column
       return false;
      }
      if ( Math.abs(col[i] - col[count]) == count - i){ // diagonal
        return false;
      }
   }
   return true;
 
 }
 
 public void solve(int n){
    int[] col = new int[n];
    func(0, col);
 }
 

Test Results:
Given n=4, it will give the answers: 2 4 1 3 and 3 1 4 2
Notes:

1. We pass the intermediate value in col[] to the next call.
2. col[] also holds the final results.
3. Algorithm books call the functions like "goodSoFar" as backtracking.

Permutation Problem

Problem: Print out all the permutations of some letters.
Inputs: An array of characters inChars of length n, where n is a positive integer
Outputs: All possible permutations of these characters.
Solution 1:

 public void func(int count, boolean[] used, char[] chs, StringBuilder sb){
    if ( count == used.length){
       System.out.println(sb.toString());
    }
    for(int i=0; i<chs.length; i++){
       if( !used[i]){
         sb.append(chs[i]);
         used[i] = true;
         func(count+1, used, chs, sb);
         used[i] = false;
         sb.setLength(count);
       }
    }
 }
 
 public void solve(char[] inChars){
   boolean[] used = new boolean[inChars.length];
   StringBuilder sb = new StringBuilder();
   func(0, used, inChars, sb);
 }
 

Test Results:
For inChars = {'a','b','c'}, the output is

   abc
   acb
   bac
   bca
   cab
   cba

Notes:

1. The above code for permutation is a little different from the n-Queens problem. In the n-Queens problem, each subsequent loop still goes throug all the positions, whereas here, the subsequent loops only go through partial values. They use the used[] array to check if a value has already been used.

Solution 2: Solution 1 is good but a little bit convoluted. We can actually use the exact same pattern as in the n-Queens problem to sovle the problem. The code is below:

 public void func(int count, int[] result, char[] chs) {
   if (count == result.length) {
    printResult(result, chs);
    return;
   }
   for (int i = 0; i < chs.length; i++) {
    result[count] = i;
    if (goodSoFar(count, result)) {
     func(count + 1, result, chs);
    }
   }
  }
  
  private boolean goodSoFar(int count, int[] used) {
   for (int i = 0; i < count; i++) {
    if (used[i] == used[count]) {
     return false;
    }
   }
   return true;
  }
  
  private void printResult(int[] result, char[] chs) {
   for (int i = 0; i < result.length; i++) {
    System.out.print(chs[result[i]]);
   }
   System.out.println();
  }
  
  public void solve(char[] inChars) {
   int[] result = new int[inChars.length];
   for (int i = 0; i < result.length; i++) {
    result[i] = -1;
   }
   func(0, result, inChars);
 }
 

Test Results:
For inChars = {'a','b','c'}, the output is

    abc
    acb
    bac
    bca
    cab
    cba

Notes:

1. The code is very similar to the n-Queens problem. We store the intermediate results in the array "result" and pass it to the next recursive call. The "result" array stores the index of the char in the original input "inChars" array.
2. Again, the "goodSoFar" method is our pruning method. It can be modified for other variations of permutation. For example, if no duplicate character is allowd, we can change the code.
3. This algo is a little bit less efficient than solution 1 because the method "goodSoFar" needs to go through an array to check values. But the code is very clear and easy to understand.

Combination Problem

Inputs: An array of characters inChars of length n, where n is a positive integer
Outputs: All subsets except the empty subset. There should be 2^n - 1 subsets.
Solution:

  public void func(int count, boolean[] flags, char[] chs){
      if ( count == flags.length){
         printResult(flags, chs);
         return;
      }
      flags[count] = true;
      func(count + 1, flags, chs);
      flags[count] = false;
      func(count + 1, flags, chs);
  }
  
  private void printResult(boolean[] flags, char[] chs){
    for(int i=0; i<flags.length; i++){
      if ( flags[i]){
           System.out.print(chs[i]);
      }
    }
    System.out.println();
  }
  
  public void solve(char[] inChars){
     boolean[] flags = new boolean[inChars.length];
     func(0,flags, inChars); 
  }
  

Test Results:
For inChars = {'a','b','c'}, the output is

abc
ab
ac
a
bc
b
c

Notes:

1. To form a subset, an element in the original set is either selected or not selected. We use "flags" in the code to denote whether each element is selected or not.
2. The "func" method does not have a for-loop as in the previous examples. But essentially it is looping through the array {true, false}.

Knapsack Problem

Problem: Given a collection of items, each with its weight and value, find a subset of these items so that the total value of the items in this subset is a max value under the condition that their total weight does not exceed a given limit.
Inputs: int[] weights, int[] values, int weightLimit
Output: int maxValue and the items shosen
Solution: This is essentially a combination problem. The code is below.

 public class Knapsack(){

 private int currentMaxValue;
 private boolean[] chosen;
 private boolean[] selected;

 public void func(int count, final int weightLimit, final int[] weights,
   final int[] values, int weightSoFar, int valueSoFar,
   boolean[] selected) {
  if (count == selected.length) {
   if (valueSoFar > currentMaxValue) {
    updateResult(valueSoFar, selected);
   }
   return;
  }

  selected[count] = true;

  // if next item makes it overweight,stop
  if (weightSoFar + weights[count] > weightLimit) {
   if (valueSoFar > currentMaxValue) {
    updateResult(valueSoFar, selected);
   }
  } else {
   func(count + 1, weightLimit, weights, values, weightSoFar
     + weights[count], valueSoFar + values[count], selected);
  }

  selected[count] = false;
  func(count + 1, weightLimit, weights, values, weightSoFar, valueSoFar,
    selected);

 }

 private void updateResult(int valueSoFar, boolean[] flags) {
  this.currentMaxValue = valueSoFar;
  // can also use the following System.arraycopy instead of the for-loop
  // System.arraycopy(flags, 0, chosen, 0, flags.length);
  for (int i = 0; i < flags.length; i++) {
   chosen[i] = flags[i];
  }
 }

 public void solve(int[] weights, int[] values, int weightLimit) {
  Assert.isTrue(weights != null && values != null
    && weights.length == values.length);
  currentMaxValue = Integer.MIN_VALUE;
  chosen = new boolean[weights.length];
  selected = new boolean[weights.length];
  func(0, weightLimit, weights, values, 0, 0, selected);
  System.out.println("The max value is " + currentMaxValue + " chosen is "
    + Arrays.toString(chosen));
 }
}

Test Result:

 public void testKnapSack() {
  int limit = 20;
  int[] weights = { 2, 5, 23, 10, 6, 7, 1 };
  int[] values = { 2, 343, 234, 1, 21, 6, 8 };
  solve(weights, values, limit);
 }
Output:

Max values is 378 chosen is [false, true, false, false, true, true, true]

The above results mean that items at the positions 2,5,6,7 are chosen. 

Notes on Character Encoding and Unicode

We often see terminologies such as ASCII, UTF-8, Unicode, etc. These can be confusing. Here are some notes on them.

1. Basically an encoding is about how to represent language characters in computer using numbers.
2. ASCII is basically for English. It uses 7 bits. Each value from 0 to 127 is mapped to a character. We know that this is not enough for other languages.
3. ISO 8859 is another encoding. It uses 8 bits. It can be thought of as an expansion of ASCII. It uses the same values to represent the same ASCII characters. With the additional one bit, it can represent more characters and is sufficient for most other western languages.
4. Unicode also maps characters to numbers (called code points). It uses 32 bits. But the range of the values is not all the 32 bit numbers. The range is from 0x0 to 0x10FFFF. For every character that has an ASCII value, the Unicode code point and the ASCII value of that character are the same.
5. Unicode is a specification. It is not an implementation. Actually the same can be said of ASCII. UTF-8 and UTF-16 are two of the Unicode implementations.
6. UTF-8 uses 8 bits to represent some characters. And uses up to 32 bits to represent other characters. It is a variable-length encoding. UTF-16 uses 16 bits to represent some characters. And uses 32 bits to represent other characters. It is also variable-length encoding. So the number 8 and 16 in these two encoding scheme can be thought of as the least number of digits used to encode a character.
7. Java uses Unicode for String. And it uses UTF-16. In java, you can use escape character to represent a Unicode character. For example you can use \u0078 for 'x'. Now immediately you can see that this format has the largest possible value \uFFFF, which does not cover the all the values in a Unicode. The largest Unicode value is 0x10FFFF. For this java uses a surrogate pair which is a combination of two \u numbers.
8. In java, you can also use another format to represent a char value. For example,

   char var = '\101'; 

   Here the variable var is equal to 'A'. The formal name for this representation is "octal escapes". The maximum value allowed in Java is '\377', which is equivalent to the decimal value 255 because 3*8^2 + 7*8 + 7 = 255. It is probably for purely historical reasons that Java supports octal escape sequences at all. These escape sequences originated in C.
9. Note that the Unicode representation in java is preprocessed before compilation. So for example "\u0033\u0078\u1000\u0079" is a valid string that consists of 4 letters "3x?y".