解决问题JDBC/ODBC访问Azure数据湖存储Gen2

学习解答JDBC和ODBC访问Azure数据湖存储Gen2

写由亚当巴夫拉卡

2022年6月1日

问题

删除

信息学

通常,当您想访问Azure数据湖存储Gen2时,应使用Databricksuntime5.2或以上,其中包括内置AzureBlob文件系统驱动文章适用于使用JDBC/ODBC存取ADLSGen2存储器的用户

sql查询JDBC或ODBC客户端访问ADLS Gen2时发生下列错误:

com.google.common.util.concurrent.UncheckedExecutionException: java.lang.IllegalArgumentException: No value for dfs.adls.oauth2.access.token.provider found in conf file.18/10/23 21:03:28 ERROR SparkExecuteStatementOperation: Error executing query, currentState RUNNING,  java.util.concurrent.ExecutionException: java.io.IOException: There is no primary group for UGI (Basic token)chris.stevens+dbadmin (auth:SIMPLE)    at com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)    at com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)    at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)    at com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:135)    at com.google.common.cache.LocalCache$Segment.getAndRecordStats(LocalCache.java:2344)    at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2316)    at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2278)    at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2193)    at com.google.common.cache.LocalCache.get(LocalCache.java:3932)    at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4721)    at org.apache.spark.sql.catalyst.catalog.SessionCatalog.getCachedPlan(SessionCatalog.scala:158)    at org.apache.spark.sql.execution.datasources.FindDataSourceTable.org$apache$spark$sql$execution$datasources$FindDataSourceTable$$readDataSourceTable(DataSourceStrategy.scala:257)    at org.apache.spark.sql.execution.datasources.FindDataSourceTable$$anonfun$apply$2.applyOrElse(DataSourceStrategy.scala:313)    at    at scala.collection.LinearSeqOptimized$class.foldLeft(LinearSeqOptimized.scala:124)    at scala.collection.immutable.List.foldLeft(List.scala:84)    at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:87)    at org.apache.spark.sql.catalyst.rules.RuleExecutor$$anonfun$execute$1.apply(RuleExecutor.scala:79)

sql客户端运行查询时有下列错误:

SQL命令执行时出错:从测试数据bricks限制50错误代码:0, SQL状态:com.google.common.util.convent.uncertExceptition:com.databricks.SQL状态=HY000,DB错误码=50051警告:[Simba][sparkJDBCDriver](500100)错误从数据库获取表信息

因果

根由错误配置设置创建JDBC或ODBC连接ABFS-ADLSgen2

求解

集成spark.hadoop.hive.server2.enable.doAs虚伪集群配置设置

文章有帮助吗