Spark 用AnyFunSuite单元测试Scala详细教程

在用java开发时，通过用Junit框架来测试，在用spark开发scala时，除了可以用Junit，还可以用AnyFunSuite，无需依赖AnyFunSuite。

步骤一：设置项目依赖

确保您的项目中包含了以下必要的依赖：

    <dependency>
      <groupId>org.apache.spark</groupId>
      <artifactId>spark-sql_2.11</artifactId>
      <version>2.4.0</version>
    </dependency>
 
<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.11</artifactId>
  <version>2.4.0</version>
  <scope>test</scope>
</dependency>
 
    <!-- ScalaTest 依赖 -->
    <dependency>
      <groupId>org.scalatest</groupId>
      <artifactId>scalatest_2.11</artifactId>
      <version>3.2.9</version>
      <scope>test</scope>
    </dependency>

步骤二、编写单元测试

例如下面wordcount的代码

import org.apache.spark.sql.SparkSession
 
object WordCount {
  def wordCount(input: String): Long = {
    val spark = SparkSession.builder().appName("WordCount").master("local[*]").getOrCreate()
    val words = spark.sparkContext.parallelize(input.split(" "))
    val count = words.count()
    spark.stop()
    count
  }
}

编写单元测试的代码：

import org.scalatest.funsuite.AnyFunSuite
 
class WordCountTest extends AnyFunSuite {
 
  test("wordCount should return correct word count") {
    val input = "Hello world, hello Scala"
    val expectedResult = 5
    val result = WordCount.wordCount(input)
    assert(result == expectedResult)
  }
 
}

步骤三：运行单元测试

在 IDEA 中右键点击测试类名或测试方法名，选择 "Run WordCountTest" 或 "Run 'wordCount should return correct word count'" 来运行单元测试。您也可以点击绿色的三角形按钮执行所有测试用例。