【无标题】

Exception in thread "main" org.apache.hadoop.security.AccessControlException: Permission denied: user=S, access=WRITE, inode="/user/hudi":root:supergroup:drwxr-xr-x

hadoop fs -chmod -R 777 /user/hudi

2

 org.apache.spark.serializer.KryoSerializer是Hudi所需的序列化器

hoodie only support org.apache.spark.serializer.KryoSerializer as spark.serializer

val spark = SparkSession.builder()
  .appName("JsonDataGenerator")
  .master("local")  .config("spark.serializer", "org.apache.spark.serializer.KryoSerializer")

3

Exception in thread "main" org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/hudi/.hoodie/hoodie.properties could only be written to 0 of the 1 minReplication nodes. There are 3 datanode(s) running and 3 node(s) are excluded in this operation.

1 hdfs dfsadmin -report

2 hdfs getconf -confKey dfs.replication检查当前副本因子配置

3 hdfs cacheadmin -addPool mycache 添加缓存

4 hdfs cacheadmin -modifyPool -pool mycache -expiryMs 86400000设置缓存池过期时间为一天

4

Exception in thread "main" java.lang.IllegalArgumentException: hoodie. properties file seems invalid.  Please check for left over `. updated` files if any, manually copy it to hoodie. properties and retry

hoodie. properties 文件有问题，去找好的替掉