使用復(fù)雜類型查詢SparkSQLDataFrame如何使用映射/數(shù)組等復(fù)雜類型查詢RDD?例如,當(dāng)我編寫這個(gè)測(cè)試代碼時(shí):case class Test(name: String, map: Map[String, String])val map = Map("hello" -> "world", "hey" -> "there")val map2 = Map("hello" ->
"people", "hey" -> "you")val rdd = sc.parallelize(Array(Test("first", map), Test("second", map2)))我想語法應(yīng)該是這樣的:sqlContext.sql("SELECT * FROM rdd WHERE map.hello = world")或sqlContext.sql("SELECT * FROM rdd WHERE map[hello] = world")但我得到了無法訪問MapType類型中的嵌套字段(StringType,true)和org.apache.spark.sql.catalyst.errors.package$TreeNodeException:未解析屬性分別。
使用復(fù)雜類型查詢SparkSQLDataFrame
qq_遁去的一_1
2019-06-17 17:29:14