问题描述
项目中使用log4j将日志写入flume,用的是flume-ng-log4jappender
包。
由于项目是Spark项目,要将项目打包后,在集群中spark-submit提交。集群中已经有了log4j、slf4j、flume等相关包,所以相关依赖都没有打包进去,直接使用集群中的。
spark-submit后报如下错:
Failed to instantiate SLF4J LoggerFactoryReported exception:java.lang.NullPointerExceptionat org.slf4j.LoggerFactory.reportActualBinding(LoggerFactory.java:344)at org.slf4j.LoggerFactory.bind(LoggerFactory.java:152)at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)at org.apache.flume.api.AbstractRpcClient.<clinit>(AbstractRpcClient.java:31)at java.lang.Class.forName0(Native Method)at java.lang.Class.forName(Class.java:264)at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:76)at org.apache.flume.clients.log4jappender.LoadBalancingLog4jAppender.activateOptions(LoadBalancingLog4jAppender.java:125)at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:217)at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:122)at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:111)at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:771)at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:102)at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:771)at org.apache.spark.internal.Logging$class.log(Logging.scala:49)at org.apache.spark.deploy.yarn.ApplicationMaster$.log(ApplicationMaster.scala:771)at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:786)at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)Exception in thread "main" java.lang.ExceptionInInitializerErrorat java.lang.Class.forName0(Native Method)at java.lang.Class.forName(Class.java:264)at org.apache.flume.api.RpcClientFactory.getInstance(RpcClientFactory.java:76)at org.apache.flume.clients.log4jappender.LoadBalancingLog4jAppender.activateOptions(LoadBalancingLog4jAppender.java:125)at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)at org.apache.log4j.PropertyConfigurator.configureRootCategory(PropertyConfigurator.java:648)at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:514)at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)at org.slf4j.impl.Log4jLoggerFactory.<init>(Log4jLoggerFactory.java:66)at org.slf4j.impl.StaticLoggerBinder.<init>(StaticLoggerBinder.java:72)at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:45)at org.apache.spark.internal.Logging$.org$apache$spark$internal$Logging$$isLog4j12(Logging.scala:217)at org.apache.spark.internal.Logging$class.initializeLogging(Logging.scala:122)at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:111)at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:771)at org.apache.spark.internal.Logging$class.initializeLogIfNecessary(Logging.scala:102)at org.apache.spark.deploy.yarn.ApplicationMaster$.initializeLogIfNecessary(ApplicationMaster.scala:771)at org.apache.spark.internal.Logging$class.log(Logging.scala:49)at org.apache.spark.deploy.yarn.ApplicationMaster$.log(ApplicationMaster.scala:771)at org.apache.spark.deploy.yarn.ApplicationMaster$.main(ApplicationMaster.scala:786)at org.apache.spark.deploy.yarn.ApplicationMaster.main(ApplicationMaster.scala)Caused by: java.lang.IllegalStateException: Unexpected initialization failureat org.slf4j.LoggerFactory.bind(LoggerFactory.java:179)at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)at org.apache.flume.api.AbstractRpcClient.<clinit>(AbstractRpcClient.java:31)... 27 moreCaused by: java.lang.NullPointerExceptionat org.slf4j.LoggerFactory.reportActualBinding(LoggerFactory.java:344)at org.slf4j.LoggerFactory.bind(LoggerFactory.java:152)... 32 more
排错过程
Caused by: java.lang.NullPointerException
报个空指针,一脸蒙蔽。跟着栈轨迹走了一圈,发现几乎不可能报空指针。于是猜测可能是jar包冲突、或者组件不匹配吧。
于是开始搜SLF4包冲突问题,调了调jar包版本,还是不行。
然后开始看组件间的匹配问题,去maven那边看依赖,对应的jar包版本也都对。崩溃了。
崩溃之余,又改了改slf4j-log4j12
的版本,改了多次之后发现竟然成功了。。。。
解决办法
集群中的jar包及对应的版本:
flume-ng-log4jappender-1.9.0-cdh6.3.2.jarflume-ng-sdk-1.9.0-cdh6.3.2.jarlog4j-1.2.17.jarslf4j-api-1.7.25.jarslf4j-log4j12-1.7.25.jar
明明上面版本就是完美契合的。。。不明白为什么不行。
将slf4j-log4j12
的版本从1.7.25改到1.7.10,就可以跑起来了。
如果觉得《slf4j-log4j12版本问题导致的NullPointerException》对你有帮助,请点赞、收藏,并留下你的观点哦!