我正在嘗試運(yùn)行這個(gè)項(xiàng)目,我在 sbt 文件中添加了依賴項(xiàng),我的 sbt 文件如下所示:name := "HelloScala"version := "0.1"scalaVersion := "2.11.8"libraryDependencies += "org.apache.spark" %% "spark-core" % "2.3.1"resolvers += Resolver.bintrayRepo("salesforce", "maven")libraryDependencies += "com.salesforce.transmogrifai" %% "transmogrifai-core" % "0.3.4"然后我Helloworld從他們的存儲(chǔ)庫(kù)中復(fù)制了該文件夾,但有很多問(wèn)題。Information:10/09/18, 12:01 PM - Compilation completed with 88 errors and 0 warnings in 15 s 624 msError:scalac: missing or invalid dependency detected while loading class file 'package.class'.Could not access type Vector in value org.apache.spark.ml.linalg,because it (or its dependencies) are missing. Check your build definition formissing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)A full rebuild may help if 'package.class' was compiled against an incompatible version of org.apache.spark.ml.linalg.Error:scalac: missing or invalid dependency detected while loading class file 'OPVector.class'.Could not access type Vector in value org.apache.spark.ml.linalg,because it (or its dependencies) are missing. Check your build definition formissing or conflicting dependencies. (Re-run with `-Ylog-classpath` to see the problematic classpath.)A full rebuild may help if 'OPVector.class' was compiled against an incompatible version of org.apache.spark.ml.linalg.Error:scalac: missing or invalid dependency detected while loading class file 'OpEvaluatorBase.class'.我試圖搜索這些問(wèn)題,發(fā)現(xiàn)它可能是版本問(wèn)題,但如果存在版本問(wèn)題,我不知道應(yīng)該使用哪個(gè)版本。但是,如果我嘗試從命令行運(yùn)行它,它會(huì)起作用:cd helloworld./gradlew compileTestScala installDist./gradlew -q sparkSubmit -Dmain=com.salesforce.hw.OpTitanicSimple -Dargs="\`pwd`/src/main/resources/TitanicDataset/TitanicPassengersTrainData.csv"它不適用于 IntelliJ ,我該如何解決這個(gè)問(wèn)題?
1 回答

呼啦一陣風(fēng)
TA貢獻(xiàn)1802條經(jīng)驗(yàn) 獲得超6個(gè)贊
在build.sbt兩個(gè)依賴是缺失的:spark-mllib和spark-sql
libraryDependencies ++= Seq(
"org.apache.spark" %% "spark-core" % "2.3.1",
"org.apache.spark" %% "spark-mllib" % "2.3.1",
"org.apache.spark" %% "spark-sql" % "2.3.1",
"com.salesforce.transmogrifai" %% "transmogrifai-core" % "0.3.4"
)
這將刪除第一個(gè)錯(cuò)誤塊。
添加回答
舉報(bào)
0/150
提交
取消