【spark内置函数】current_date、current_timestamp、date_format

女爷i 2022-09-08 05:57 274阅读 0赞

**一、目的**

演示sparkSql三种内置函数current\_date、current\_timestamp、date\_format的使用；

注意：current\_timestamp输出的并不是长整型时间戳，而是2021-08-25 17:35:25.568格式；

**二、代码**

import org.apache.spark.sql.{DataFrame, SparkSession}
    
    object Test {
      case class DateT(name:String,birthday:String)
      def main(args: Array[String]): Unit = {
        val spark = SparkSession.builder().master("local[*]").appName("testDate").getOrCreate()
        import spark.implicits._
        import org.apache.spark.sql.functions._
        val sc = spark.sparkContext
        sc.setLogLevel("WARN")
    
        val DateTest: DataFrame = Seq(
          DateT("aa","1995-12-11 12:12:13"),
          DateT("bb","2000-01-14 10:10:57")
        ).toDF()
    
        DateTest.withColumn("current_date",current_date())
          .withColumn("current_timestamp",current_timestamp())
          .withColumn("date",date_format(current_timestamp(),"yyyy-MM-dd HH:mm:ss"))
          .withColumn("date1",date_format(current_timestamp(),"yyyy-MM-dd hh:mm:ss"))
          .withColumn("date2",date_format(col("birthday"),"yyyy-MM-dd"))
          .show(false)
    
        spark.close()
      }
    }

**三、输出**

+----+-------------------+------------+-----------------------+-------------------+-------------------+----------+
    |name|birthday           |current_date|current_timestamp      |date               |date1              |date2     |
    +----+-------------------+------------+-----------------------+-------------------+-------------------+----------+
    |aa  |1995-12-11 12:12:13|2021-08-25  |2021-08-25 17:35:25.568|2021-08-25 17:35:25|2021-08-25 05:35:25|1995-12-11|
    |bb  |2000-01-14 10:10:57|2021-08-25  |2021-08-25 17:35:25.568|2021-08-25 17:35:25|2021-08-25 05:35:25|2000-01-14|
    +----+-------------------+------------+-----------------------+-------------------+-------------------+----------+