當(dāng)我嘗試創(chuàng)建數(shù)據(jù)集并從中收集數(shù)據(jù)時,我遇到了一個奇怪的異常。Exception in thread "main" java.lang.NumberFormatException: Zero length BigInteger at java.math.BigInteger.<init>(BigInteger.java:302) at org.apache.spark.sql.catalyst.expressions.UnsafeRow.getDecimal(UnsafeRow.java:405) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.writeFields_3_3$(generated.java:298) at org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificUnsafeProjection.apply(generated.java:35) at org.apache.spark.sql.execution.LocalTableScanExec$$anonfun$unsafeRows$1.apply(LocalTableScanExec.scala:41) at org.apache.spark.sql.execution.LocalTableScanExec$$anonfun$unsafeRows$1.apply(LocalTableScanExec.scala:41) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234) at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)執(zhí)行代碼如下所示:List<SimplePojo> list ...Dataset<SimplePojo> ds = sparkSession.createDataset(list, Encoders.bean(SimplePojo.class))ds.collectoAsList();SimplePojo 類包含一個方法 getSomething(),這顯然會導(dǎo)致異常。當(dāng)我注釋掉它時,一切都很好。public class SimplePojo { private int id; private OtherPojo otherPojo = new OtherPojo(); @Deprecated // required by park serialization. Use builder public SimplePojo(){} publi int getId(){ return id; } public String getSomething() { return otherPojo.getSomething(); } // sets ...}有沒有人知道可能發(fā)生了什么?
1 回答

溫溫醬
TA貢獻1752條經(jīng)驗 獲得超4個贊
事實證明,我不能有不屬于豆子的get。我假設(shè)它是分析器期望的“豆類格式”。Introspector在發(fā)現(xiàn)一個沒有各自屬性的get時會發(fā)瘋(根據(jù)Java版本的不同方式)。
由于我無論如何都找不到對分析器說“忽略此方法”,因此解決方案是避免與bean屬性無關(guān)的get和set。
添加回答
舉報
0/150
提交
取消