public class BQGL {
public static void main(String[] args) {
SparkConf conf = new SparkConf().setAppName("MyApp!!").setMaster("local");
JavaSparkContext sc = new JavaSparkContext(conf);
SQLContext sqlContext = new SQLContext(sc);
sc.setLogLevel("ERROR");
//将sdb中的表映射成spark中的表
sqlContext.sql("CREATE temporary table personNameAge ( name string, age int ) " +
"using com.sequoiadb.spark " +
"OPTIONS ( host 'node1:11810', " +
"collectionspace 'scott', " +
"collection 'personNameAge')");
sqlContext.sql("CREATE temporary table personNameScore ( name string, score int ) " +
"using com.sequoiadb.spark OPTIONS ( host 'node1:11810', " +
"collectionspace 'scott', " +
"collection 'personNameScore')");
//将sql转换成dataframe
DataFrame df1 = sqlContext.sql("select * from personNameAge a,personNameScore b " +
"where a.name=b.name ");
df1.printSchema();
df1.show();
请教,我要将df1保存到巨杉中怎么处理,SequoiadbWriter的save方法纠结了好久不会用。
def save(it: Iterator[Row], schema: StructType):
参数中的Iterator[Row]不知到怎么来。
http://www.sequoiadb.com/cn/index.php?a=index&m=Files&cat_id=1432190712&edition_id=0
这个页面只展示了怎么读,能否提供下怎么写的脚本案例?或者有没有其他的方法。多谢了。