在Spark中連接MySQL數據庫有兩種方式:
import org.apache.spark.sql.SparkSession
val spark = SparkSession.builder()
.appName("MySQLExample")
.getOrCreate()
val url = "jdbc:mysql://hostname:port/databaseName"
val table = "tableName"
val properties = new Properties()
properties.put("user", "username")
properties.put("password", "password")
val df = spark.read.jdbc(url, table, properties)
df.show()
首先需要在spark-submit
命令中添加MySQL Connector的jar包路徑:
spark-submit --jars /path/to/mysql-connector-java.jar --class your_class your_jar.jar
然后在代碼中使用MySQL Connector連接MySQL數據庫:
import org.apache.spark.sql.SparkSession
val spark = SparkSession.builder()
.appName("MySQLExample")
.getOrCreate()
val url = "jdbc:mysql://hostname:port/databaseName"
val table = "tableName"
val properties = new Properties()
properties.put("user", "username")
properties.put("password", "password")
val df = spark.read.format("jdbc")
.option("url", url)
.option("dbtable", table)
.option("user", properties.getProperty("user"))
.option("password", properties.getProperty("password"))
.load()
df.show()
以上是兩種連接MySQL數據庫的方式,可以根據需要選擇適合自己的方法。