【Springboot】——整合p6spy格式化SQL日志

r囧r小猫 2022-04-17 02:14 904阅读 0赞

#  项目现状介绍 #

项目使用Springboot+Mybatis作为基础框架，项目中日志框架采用logback+Slf4J，关于日志中SQL部分日志的打印采用的mybatis本身打印格式，sql语句正常输出，参数的位置使用占位符“？”替代。这样造成问题就是在处理项目中的sql问题，sql格式打印不规范，从sql日志查看，到sql日志运行判断都会花费一些重复的时间，既然这样那就想办法节省这一部分的时间吧。在Java项目中格式化sql日志，推荐使用辅助jar包，p6spy。

# 何为p6spy？ #

p6spy是针对数据库访问操作的动态监测框架（开源项目）它使得数据库数据可无缝截取和操纵，而不必对现有应用程序的代码作任何修改。P6Spy 分发包包括P6Log，它是一 个可记录任何 Java 应用程序的所有JDBC事务的应用程序。其配置完成使用时，可以进行数据访问性能的监测。  
  p6spy是一个可以用来在应用程序中拦截和修改数据操作语句的开源框架。 通过p6spy我们可以对SQL语句进行拦截，相当于一个SQL语句的记录器，这样我们可以用它来作相关的分析，比如性能分析等我们最需要的功能，查看sql语句，不是预编译的带问号的，而是真正的数据库执行的sql，更直观，更简单。  
p6spy相关材料的获取途径  
项目首页：[http://p6spy.github.io/p6spy/][http_p6spy.github.io_p6spy]  
GitHub下载介绍页面：[https://github.com/p6spy/p6spy/wiki/Download][https_github.com_p6spy_p6spy_wiki_Download]  
GitHub托管地址：[https://github.com/p6spy/p6spy/tree/master][https_github.com_p6spy_p6spy_tree_master]  
帮助文档地址：[http://p6spy.readthedocs.io/en/latest/][http_p6spy.readthedocs.io_en_latest]

# Springboot+Mybatis整合p6spy #

#引入jar包
    <dependency>
            <groupId>p6spy</groupId>
            <artifactId>p6spy</artifactId>
            <version>3.7.0</version>
    </dependency>

###引入配置文件spy.properties
    ###
    # #%L
    # P6Spy
    # %%
    # Copyright (C) 2013 P6Spy
    # %%
    # Licensed under the Apache License, Version 2.0 (the "License");
    # you may not use this file except in compliance with the License.
    # You may obtain a copy of the License at
    #
    #      http://www.apache.org/licenses/LICENSE-2.0
    #
    # Unless required by applicable law or agreed to in writing, software
    # distributed under the License is distributed on an "AS IS" BASIS,
    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    # See the License for the specific language governing permissions and
    # limitations under the License.
    # #L%
    ###
    #################################################################
    # P6Spy Options File                                            #
    # See documentation for detailed instructions                   #
    # http://p6spy.github.io/p6spy/2.0/configandusage.html          #
    #################################################################
    #################################################################
    # MODULES                                                       #
    #                                                               #
    # Module list adapts the modular functionality of P6Spy.        #
    # Only modules listed are active.                               #
    # (default is com.p6spy.engine.logging.P6LogFactory and         #
    # com.p6spy.engine.spy.P6SpyFactory)                            #
    # Please note that the core module (P6SpyFactory) can't be      #
    # deactivated.                                                  #
    # Unlike the other properties, activation of the changes on     #
    # this one requires reload.                                     #
    #################################################################
    #modulelist=com.p6spy.engine.spy.P6SpyFactory,com.p6spy.engine.logging.P6LogFactory,com.p6spy.engine.outage.P6OutageFactory
    ################################################################
    # CORE (P6SPY) PROPERTIES                                      #
    ################################################################
    # A comma separated list of JDBC drivers to load and register.
    # (default is empty)
    #
    # Note: This is normally only needed when using P6Spy in an
    # application server environment with a JNDI data source or when
    # using a JDBC driver that does not implement the JDBC 4.0 API
    # (specifically automatic registration).
    #driverlist=
    #####配置数据库驱动，根据自己的数据库选择
    driverlist=com.mysql.jdbc.Driver
    # for flushing per statement
    # (default is false)
    #autoflush = false
    # sets the date format using Java's SimpleDateFormat routine.
    # In case property is not set, miliseconds since 1.1.1970 (unix time) is used (default is empty)
    #dateformat=
    # prints a stack trace for every statement logged
    #stacktrace=false
    # if stacktrace=true, specifies the stack trace to print
    #stacktraceclass=
    # determines if property file should be reloaded
    # Please note: reload means forgetting all the previously set
    # settings (even those set during runtime - via JMX)
    # and starting with the clean table
    # (default is false)
    #reloadproperties=false
    reloadproperties=true
    # determines how often should be reloaded in seconds
    # (default is 60)
    #reloadpropertiesinterval=60
    # specifies the appender to use for logging
    # Please note: reload means forgetting all the previously set
    # settings (even those set during runtime - via JMX)
    # and starting with the clean table
    # (only the properties read from the configuration file)
    # (default is com.p6spy.engine.spy.appender.FileLogger)
    ###选择sql日志输出的问题 slf4jLogger输出到控制台，fileLogger输出到文件中
    appender=com.p6spy.engine.spy.appender.Slf4JLogger
    #appender=com.p6spy.engine.spy.appender.StdoutLogger
    #appender=com.p6spy.engine.spy.appender.FileLogger
    # name of logfile to use, note Windows users should make sure to use forward slashes in their pathname (e:/test/spy.log)
    # (used for com.p6spy.engine.spy.appender.FileLogger only)
    # (default is spy.log)
    ###如果选择输出到文件中自定义文件的输出路径
    logfile = /data/www/logs/spy.log
    # append to the p6spy log file. if this is set to false the
    # log file is truncated every time. (file logger only)
    # (default is true)
    #append=true
    # class to use for formatting log messages (default is: com.p6spy.engine.spy.appender.SingleLineFormat)
    #logMessageFormat=com.p6spy.engine.spy.appender.MultiLineFormat
    #自定义日志格式，在类中定义，因为原生的p6spy日志格式虽然取代了“？”但是全部输出到一行，可读性差，还是建议自定义一下
    logMessageFormat=com.xxx.xxxx.common.datasource.P6Spy
    #logMessageFormat=com.p6spy.engine.spy.appender.SingleLineFormat
    # format that is used for logging of the date/time/... (has to be compatible with java.text.SimpleDateFormat)
    # (default is dd-MMM-yy)
    #databaseDialectDateFormat=dd-MMM-yy
    ##输出时间格式
    databaseDialectDateFormat=yyyy-MM-dd HH:mm:ss
    # whether to expose options via JMX or not
    # (default is true)
    #jmx=true
    # if exposing options via jmx (see option: jmx), what should be the prefix used?
    # jmx naming pattern constructed is: com.p6spy(.<jmxPrefix>)?:name=<optionsClassName>
    # please note, if there is already such a name in use it would be unregistered first (the last registered wins)
    # (default is none)
    #jmxPrefix=
    #################################################################
    # DataSource replacement                                        #
    #                                                               #
    # Replace the real DataSource class in your application server  #
    # configuration with the name com.p6spy.engine.spy.P6DataSource #
    # (that provides also connection pooling and xa support).       #
    # then add the JNDI name and class name of the real             #
    # DataSource here                                               #
    #                                                               #
    # Values set in this item cannot be reloaded using the          #
    # reloadproperties variable. Once it is loaded, it remains      #
    # in memory until the application is restarted.                 #
    #                                                               #
    #################################################################
    #realdatasource=/RealMySqlDS
    #realdatasourceclass=com.mysql.jdbc.jdbc2.optional.MysqlDataSource
    #################################################################
    # DataSource properties                                         #
    #                                                               #
    # If you are using the DataSource support to intercept calls    #
    # to a DataSource that requires properties for proper setup,    #
    # define those properties here. Use name value pairs, separate  #
    # the name and value with a semicolon, and separate the         #
    # pairs with commas.                                            #
    #                                                               #
    # The example shown here is for mysql                           #
    #                                                               #
    #################################################################
    #realdatasourceproperties=port;3306,serverName;myhost,databaseName;jbossdb,foo;bar
    #################################################################
    # JNDI DataSource lookup                                        #
    #                                                               #
    # If you are using the DataSource support outside of an app     #
    # server, you will probably need to define the JNDI Context     #
    # environment.                                                  #
    #                                                               #
    # If the P6Spy code will be executing inside an app server then #
    # do not use these properties, and the DataSource lookup will   #
    # use the naming context defined by the app server.             #
    #                                                               #
    # The two standard elements of the naming environment are       #
    # jndicontextfactory and jndicontextproviderurl. If you need    #
    # additional elements, use the jndicontextcustom property.      #
    # You can define multiple properties in jndicontextcustom,      #
    # in name value pairs. Separate the name and value with a       #
    # semicolon, and separate the pairs with commas.                #
    #                                                               #
    # The example shown here is for a standalone program running on #
    # a machine that is also running JBoss, so the JDNI context     #
    # is configured for JBoss (3.0.4).                              #
    #                                                               #
    # (by default all these are empty)                              #
    #################################################################
    #jndicontextfactory=org.jnp.interfaces.NamingContextFactory
    #jndicontextproviderurl=localhost:1099
    #jndicontextcustom=java.naming.factory.url.pkgs;org.jboss.nameing:org.jnp.interfaces
    #jndicontextfactory=com.ibm.websphere.naming.WsnInitialContextFactory
    #jndicontextproviderurl=iiop://localhost:900
    ################################################################
    # P6 LOGGING SPECIFIC PROPERTIES                               #
    ################################################################
    # filter what is logged
    # please note this is a precondition for usage of: include/exclude/sqlexpression
    # (default is false)
    #filter=true
    # comma separated list of strings to include
    # please note that special characters escaping (used in java) has to be done for the provided regular expression
    # (default is empty)
    #include=select
    # comma separated list of strings to exclude
    # (default is empty)
    #exclude =
    # sql expression to evaluate if using regex
    # please note that special characters escaping (used in java) has to be done for the provided regular expression
    # (default is empty)
    #sqlexpression =
    #list of categories to exclude: error, info, batch, debug, statement,
    #commit, rollback and result are valid values
    # (default is info,debug,result,resultset,batch)
    #excludecategories=info,debug,result,resultset,batch
    #excludecategories=error,info,debug,result,resultset
    #excludecategories=info
    # Execution threshold applies to the standard logging of P6Spy.
    # While the standard logging logs out every statement
    # regardless of its execution time, this feature puts a time
    # condition on that logging. Only statements that have taken
    # longer than the time specified (in milliseconds) will be
    # logged. This way it is possible to see only statements that
    # have exceeded some high water mark.
    # This time is reloadable.
    #
    # executionThreshold=integer time (milliseconds)
    # (default is 0)
    #executionThreshold=
    ################################################################
    # P6 OUTAGE SPECIFIC PROPERTIES                                #
    ################################################################
    # Outage Detection
    #
    # This feature detects long-running statements that may be indicative of
    # a database outage problem. If this feature is turned on, it will log any
    # statement that surpasses the configurable time boundary during its execution.
    # When this feature is enabled, no other statements are logged except the long
    # running statements. The interval property is the boundary time set in seconds.
    # For example, if this is set to 2, then any statement requiring at least 2
    # seconds will be logged. Note that the same statement will continue to be logged
    # for as long as it executes. So if the interval is set to 2, and the query takes
    # 11 seconds, it will be logged 5 times (at the 2, 4, 6, 8, 10 second intervals).
    #
    # outagedetection=true|false
    # outagedetectioninterval=integer time (seconds)
    #
    # (default is false)
    #outagedetection=false
    # (default is 60)
    #outagedetectioninterval=30

自定义日志输出格式代码

import com.p6spy.engine.spy.appender.MessageFormattingStrategy;
    
    import java.text.SimpleDateFormat;
    import java.util.Date;
    
    public class P6Spy implements MessageFormattingStrategy {
        private SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss:SSS");
    
        @Override
        public String formatMessage(int connectionId, String now, long elapsed, String category, String prepared, String sql) {
            return !"".equals(sql.trim()) ? this.format.format(new Date()) + " | took " + elapsed + "ms | " + category + " | connection " + connectionId + "\n " + sql + ";" : "";
        }
    }

修改数据源，因为项目中配置了动态链接数据源，所以在代码中将返回Datasource的地方替换成P6Datasource

public DataSource createDataSource(DataSourceConfig dataSourceConfig, Map<String, Object> propertyMap) {
    
            Class<? extends DataSource> dataSourceType;
            try {
                dataSourceType = (Class<? extends DataSource>) Class.forName((String) "org.apache.tomcat.jdbc.pool.DataSource");
                DataSourceBuilder factory = DataSourceBuilder.create().driverClassName("com.mysql.jdbc.Driver").url(dataSourceConfig.getUrl())
                        .username(dataSourceConfig.getUserName()).password(dataSourceConfig.getPassWord()).type(dataSourceType);
                ConversionService conversionService = new DefaultConversionService();
                DataSource dataSource = factory.build();
                RelaxedDataBinder dataBinder = new RelaxedDataBinder(dataSource);
                dataBinder.setConversionService(conversionService);
                dataBinder.setIgnoreNestedProperties(false);//false
                dataBinder.setIgnoreInvalidFields(false);//false
                dataBinder.setIgnoreUnknownFields(true);//true
                PropertyValues dataSourcePropertyValues = new MutablePropertyValues(propertyMap);
                dataBinder.bind(dataSourcePropertyValues);
                return new P6DataSource(dataSource);
            } catch (ClassNotFoundException e) {
                LOGGER.error("创建数据源失败", e);
                throw new RuntimeException(e);
            }
        }

添加前的SQL，添加之后的SQL日志  
![在这里插入图片描述][20181111230126413.png]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYWRhamluZzI2Nw_size_16_color_FFFFFF_t_70]  
注意⚠️  
    p6spy本身配合slf4j+log4j使用，项目中需要整合logback，目前小编的项目还有一点不完美就是，sql.log日志文件不能自动自动切割，如果一定要利用logback来完成，p6spy打印的所有日志，输出到main函数所在的目录下。如果一定希望能自动切割，那就多关注一些main函数所在的包的日志切割情况。

# 总结 #

之前自己对技术的应用大部分停留到应用的层面，很片面的认为很少有机会去根据项目的需要改动现有框架的源码，所以会用就Ok了。但是这次在项目中整合p6spy，因为日志框架的问题，再加上对p6spy深入认知不够，还尚未完美的将p6spy+logback整合到一起。也让我反思对技术为什么要追求更高层次上的认知！

[http_p6spy.github.io_p6spy]: http://p6spy.github.io/p6spy/
[https_github.com_p6spy_p6spy_wiki_Download]: https://github.com/p6spy/p6spy/wiki/Download
[https_github.com_p6spy_p6spy_tree_master]: https://github.com/p6spy/p6spy/tree/master
[http_p6spy.readthedocs.io_en_latest]: http://p6spy.readthedocs.io/en/latest/
[20181111230126413.png]: /images/20220417/ec6e3b9dcbda4be298ae74ae37261c8b.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ppYWRhamluZzI2Nw_size_16_color_FFFFFF_t_70]: /images/20220417/ffdf08e6ad2648a6a064b023d5d50827.png