我正在嘗試運行這個項目,我在 sbt 文件中添加了依賴項,我的 sbt 文件如下所示:name := "HelloScala"version := "0.1"scalaVersion := "2.11.8"libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.1"resolvers += Resolver.bintrayRepo("salesforce", "maven")libraryDependencies += "com.salesforce.transmogrifai" %% "transmogrifai-core" % "0.3.4"然后我Helloworld從他們的存儲庫中復制了該文件夾,但有很多問題。Information:10/09/18, 12:01 PM - Compilation completed with 88 errors and 0 warnings in 15 s 624 msError:scalac: missing or invalid dependency detected while loading class file 'package.class'.Could not access type Vector in value org.apache.spark.ml.linalg,because it (or its dependencies) are missing. Check your build definition formissing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)A full rebuild may help if 'package.class' was compiled against an incompatible version of org.apache.spark.ml.linalg.Error:scalac: missing or invalid dependency detected while loading class file 'OPVector.class'.Could not access type Vector in value org.apache.spark.ml.linalg,because it (or its dependencies) are missing. Check your build definition formissing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)A full rebuild may help if 'OPVector.class' was compiled against an incompatible version of org.apache.spark.ml.linalg.Error:scalac: missing or invalid dependency detected while loading class file 'OpEvaluatorBase.class'.我試圖搜索這些問題,發(fā)現它可能是版本問題,但如果存在版本問題,我不知道應該使用哪個版本。但是,如果我嘗試從命令行運行它,它會起作用:cd helloworld./gradlew compileTestScala installDist./gradlew -q sparkSubmit -Dmain=com.salesforce.hw.OpTitanicSimple -Dargs="\`pwd`/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv"它不適用于 IntelliJ ,我該如何解決這個問題?
1 回答

呼啦一陣風
TA貢獻1802條經驗 獲得超6個贊
在build.sbt兩個依賴是缺失的:spark-mllib和spark-sql
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.3.1",
"org.apache.spark" %% "spark-mllib" % "2.3.1",
"org.apache.spark" %% "spark-sql" % "2.3.1",
"com.salesforce.transmogrifai" %% "transmogrifai-core" % "0.3.4"
)
這將刪除第一個錯誤塊。
添加回答
舉報
0/150
提交
取消