Intermittent startup failure
Description
Activity
Stephen Marquard August 2, 2020 at 10:35 AM
From startup logs with startScheduler=false:
2020-08-02 17:31:43,808 INFO main org.sakaiproject.component.app.scheduler.ScheduledInvocationManagerImpl - init()
2020-08-02 17:31:48,841 INFO main org.sakaiproject.component.app.scheduler.AutoRun - AutoRun running
2020-08-02 17:31:48,879 INFO main org.sakaiproject.component.app.scheduler.AutoRun - Triggered job: Get Archives
2020-08-02 17:31:49,403 INFO main org.sakaiproject.component.app.scheduler.SchedulerManagerImpl - Scheduler is disabled
so it also seems this AutoRun should not do anything if startScheduler is false.
Stephen Marquard August 2, 2020 at 9:56 AM
Seen on a 20.x app server (startScheduler=false)
2020-07-29 06:11:52,609 ERROR main org.sakaiproject.component.app.scheduler.SchedulerManagerImpl - Failed to start scheduler.
java.lang.NullPointerException
at org.sakaiproject.component.app.scheduler.SchedulerManagerImpl.init(SchedulerManagerImpl.java:171)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
Matthew Jones July 26, 2020 at 6:23 PM
Hmm, I looked at this again (after 2 years).
It looks like this exception was in the init(), so it would run even if startScheduler was false.
Basically what would happen is if any nodes are started up close together and one server removes a job.
This property probably should also stop the init too.
Just in case this is true and hits this exception it seems like this can be safely skipped but we should probably print up a warning about this in the logs
Austin July 21, 2020 at 8:34 PMEdited
We also hit this today on Sakai 12.7. I think this is the first time I've seen it. this has happened 3 times in the last couple of months, on two different nodes, luckily they happened in a 'test' environment. (I still need to check our prod logs, but I don't recall it happening on prod)
Like Stephen, we also have
startScheduler@org.sakaiproject.api.app.scheduler.SchedulerManager=false
on the server that failed.
What was also a little scary, but hopefully not involved is that I had just applied the patches for SAK-43292, SAK-43708, SAK-43752, which are for upgrading sitestats' jchart... but I sure hope that was a coincidence
Stephen Marquard July 24, 2018 at 12:45 PMEdited
Multiple times. Also the nodes on which this problem showed up have
startScheduler@org.sakaiproject.api.app.scheduler.SchedulerManager=false
(we only have this set to true on one node server in the cluster).
Details
Details
Priority
Major
Getting this intermittently when starting up multiple nodes. It kills the startup and tomcat must be restarted.
2018-07-22 17:53:49,401 ERROR localhost-startStop-1 org.sakaiproject.component.app.scheduler.SchedulerManagerImpl - Failed to start scheduler. java.lang.NullPointerException at org.sakaiproject.component.app.scheduler.SchedulerManagerImpl.init(SchedulerManagerImpl.java:205) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1758) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1695) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1624) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:555) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:351) at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1531) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1276) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553) at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197) at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761) at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867) at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543) at org.sakaiproject.component.impl.SpringCompMgr.init(SpringCompMgr.java:164) at org.sakaiproject.component.cover.ComponentManager.getInstance(ComponentManager.java:115) at org.sakaiproject.component.cover.ComponentManager.get(ComponentManager.java:126) at org.sakaiproject.util.ToolListener.<init>(ToolListener.java:88) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)